scroll to the bottom to the see the implemented workflow, using imgproxy

Ya it sounds not needed

We can try that. I am just afraid it’s going to be too slow

imgproxy seems to have a pretty small memory and CPU footprint

I think they compute demand based on use of cpus and memory dynamically. So I’m not sure yet if the api will use up it all

You also have the option of spinning up and down the service based on when it gets pinged , if you want to use it like a ‘server less’ service. (In service project settings )

From my understanding would it always run so it would have the maximum cost, just like the API service

Additional railway services are charged based on use , so it prob won’t cost much

Oh wow didn’t know webp could animate !

I envisioned the system originally as the card displays a small filesize image but if you click into the card and click the image preview in there then it’ll send you to the url for the original upload

I should’ve mentioned this earlier but a requirement is that the user can still fetch the original image. Is this possible w the proxy?

converts/compresses image and caches it for later

get original image

+ Only requires small client-side changes

+ does not require to change how images are uploaded and stored on S3

+ Don’t have to worry about lib issues with Sharp etc.

+ may handle huge parallel loads of resizing tasks more efficiently

+ secured against ppl using images as an attack vector

+ separates the problem of optimizaton/compression/conversion of images from rest of the app

+ Easier to maintain

+ Can quickly try different compression & size settings because we request images & runtime

+ does not require server-side changes

+ does not introduce breaking changes if the solution does not work out

how does imgproxy handle transparent gifs? and animated gifs?

Just tested it. For animated gifs it creates a smaller webp version 1,3 MB -> 126 KB

+ Don’t have to worry about lib issues with Sharp etc.

+ secured against ppl using images as an attack vector

+ may handle huge parallel loads of resizing tasks more efficiently

+ the same pipeline can be reused for other types of files

+ saves an http roundtrip

+ DRY we already have code for uploading to s3

+ easier to test, debug

responds with processed image

add url to db queue

kinopio-server updates the card record and websockets the small url to space

kinopio-server processes the file and uploads it to s3

after each user upload

imgproxy will process and upload the file to s3 , then send the processed url to kinopio-server

after each user upload,

for converting images

for converting audio and video

queue processing of original file URL

store processed file

Other (PDF, ZIP etc.)

Convert to WebP or JPEG

Pulls original file from S3 and creates optimized versions like described above in ‘Media Pipeline’

update client with new URL

Queueing via BullMQ which is backed by Redis

notify server of success

uploads original file to S3

urlOriginalFile

or if the file is a heic/tiff/wav or some other format that can’t be viewed in browsers

we only need to do img conversions in the case that the file is too large (by some threshold, like file size > 300 kb)

is there a reason to not always just use webp for images?

url to use in card name sent back to client (compressed version, or original if no compression needed)

Converted and compressed file