Transcribing/elaborating large files

[alessandrobelli]

Hello!
In our staging server testing aTrain #40 some questions are rising about file sizes.
Foremost, we should code a hard limit, such as 100 or 200 MB. Depending on the solution below, we can increase it.

What do we do with large files?

This applies only to our internal plugins.

This morning I had a chat with @hohse and there are many options:

1. always keep the uploaded file and the elaborated file.
2. keep the smaller files (than 100mb?) - send to aTrain and when it's transcribed, delete it.
   2.1. Define small files. Even with 100mb, we can quickly run out of space.
3. Always keep only the elaborated file.
   3.1 this is the "safe" way to ensure we don't run out of space.

Laravel has an easy way to plug-in any kind of storage, so S3 or others, and I can increase the storage of our openQDA server, but I guess we need to think about something for the Pre-Release: https://github.com/openqda/openqda/milestone/1

Ideally our users will be able to plug their own storage, which is doable, but far down the line.

[jankapunkt]

I voted for 3. as short-term solution but for the long run we need the files for playback so my idea is the following as long term solution:

• build a storage plugin (using an s3 compatible protocol, note that we first need specifications before we implement this!!!!!!!1!1!111!)
• uploaded files get moved to aTrain for transcription AND to a second service, that compresses the audio files
• transciptions and compressed audios are saved in the external storage and not on the openqda server

[kadewe]

Also please ask the people to upload mp3 and not WAV! Or if people upload WAV it should be FIRST encoded/converted into 192KBS Mono, the WAV should be trashed, and only the mp3 should be in later scenarios be archived