Docs

Get Started

The Sorcerer API allows you to programmatically initiate uploads to your preferred cloud storage provider. Additionally, you can use this tool to view progress and cancel uploads. If parameters are necessary for a request, the body of request must be JSON encoded. To utilize the REST API, cURL or a HTTP library is needed for your corresponding programming language. For Python, the most utilized library is requests. For Node.js, Node Fetch is a suitable library. To create an API key, go to your account and click the "API Keys" tab.

Create an upload process

POST/upload

$ curl https://api.sorcerer.io/upload \
-H "client_id: YOUR_CLIENT_ID" \
-H "client_secret: YOUR_CLIENT_SECRET" \
-H "Content-Type: application/json" \
-d '{"url":"https://wikipedia.org/fake.archive.tar.gz", "connection_id": "2022-07-15T02:28:22.047ZiyceBEGgjO"}'

RESPONSE

{
"process_id" : "2022-07-16T16:06:35.560ZbySbGCJ0vT" ,
"url" : "https://dumps.wikimedia.org/wikidatawiki/20211120/wikidatawiki-20211120-stub-meta-current27.xml.gz" ,
"step" : 1 ,
"size" : 1235162647 ,
"connection_id" : "2022-07-15T02:28:22.047ZiyceBEGgjO" ,
"connection_type" : "S3" ,
"connection_name" : "sorcerer_bucket" ,
"file_key" : "new-wikipedia-file-name.gz" ,
"folder_path" : null ,
"og_file_name" : "wikidatawiki-20211120-stub-meta-current27.xml.gz" ,
"mime_type" : "application/x-gzip" ,
"upload_id" : null ,
"completed_chunks" : 0 ,
"chunk_count" : 200 ,
"creation_date" : 1639959367 ,
"start_date" : null ,
"completion_date" : null ,
"error_message" : null ,
"stream" : {}
}

Parameters

urlREQUIRED
The url to upload.

connection_idREQUIRED
The connection ID used to define the location where the file will be uploaded.

file_keyoptional
The new name for the requested file that will overwrite the original name.

folder_pathoptional
The folder path that the file should be downloaded into. Only valid with Google Drive, Dropbox, Box, and Microsoft Onedrive connections.

random_keyoptional
Defines whether the uploaded file will keep the original filename or recieve a unique key filename. By default, this parameter is "false". This parameter is ignored if a new name is provided through the "file_key" parameter.

max_sizeoptional
The maximum size of the file that will be downloaded. If the file's size exceeds the provided figure, the process will not be submitted. Note that for stream manifest, this is compared against the estimated size since streams do not provide a file size.

min_sizeoptional
The minimum size of the file that will be downloaded. If the file's size is lower than the provided figure, the process will not be submitted. Note that for stream manifest, this is compared against the estimated size since streams do not provide a file size.

if_modified_byoptional
The latest modification date for a file submitted as a timestamp. If the provided date is less than the file's actual last modification date, the process will not be submitted.

adjust_extensionoptional
If this parameter is set to "true", the file's extension will modified to match its content type. For instance, if a file has a "text/html" mime but the extension is not ".html", Sorcerer will change the extension. The default value for this parameter is "false".

video_qualityoptional
The preferred resolution stream to download for DASH and HLS streams. If the provided resolution is not available, the next best option is downloaded. Available options include: "worst","240","360","540","720","1080", "1440", "2160", "best". By default, the 720p stream is downloaded if the resolution parameter is not provided. This parameter is ignored if the requested stream manifest only contains audio components.

video_idoptional
The preferred video format ID to download. This is only accessible using the "sniff" endpoint and this supersedes the "video_quality" parameter if both are submitted.

audio_idoptional
The preferred audio format id to download. This ID is only accessible using the "sniff" endpoint and this parameter is ignored if the requested stream manifest only contains video components. The formats "worst" and "best" are also applicable with this parameter.

subtitle_languagesoptional
An array of subtitle languages submitted in the ISO 3166-2 international standard (EX: en, pt, ru). A maximum of 10 languages can submitted to be included in the stream download process.

Returns

Returns an upload process object.

The upload process object

RESPONSE

"audio_chunk_count" : 0 ,
"audio_id" : null ,
"audio_quality" : null ,
"completed_audio_chunks" : 0 ,
"completed_video_chunks" : 0 ,
"metadata" : {} ,
"subtitle_languages" : [] ,
"video_chunk_count" : 0 ,
"video_id" : null ,
"video_quality" : null
}

}

process_idstring
The identifier of the upload process. Used to retrieve and track progress.

connection_idstring
The connection ID used to define the location where the file will be uploaded.

connection_typestring
The type of connection used.

connection_namestring
The name of the connection used.

folder_pathstring
The folder path that the file will be downloaded into.

stepinteger
The status of the upload.

Process Steps
A "1" status is directly after a process is created and when metadata is pulled.
A "2" status indicates that the upload is currently in progress.
A "3" status indicates that the media file is being transcoded.
A "4" status indicates that the transcoded media file is being uploaded.
A "5" status indicates that all chunks have been uploaded and are currently being stitched together to complete the process.
A "6" status indicates that the upload has completed.

sizeinteger
The size in bytes of the requested url.

upload_idstring
The identifier of the upload in the connection's cloud provider.

og_file_namestring
The original file name.

file_keystring
The unique name of the file when uploaded to the connection.

mime_typestring
The MIME type of the file.

chunk_countinteger
The number of chunks that will be uploaded and then combined. The size of the chunks is dependent on the size of the requested file.

completed_chunksinteger
The number of chunks that have been uploaded. This figure combined with the chunk_count can be used to check the progress of the upload.

streamobject
If the process is a stream download, this object will be filled with the relevant attributes.

creation_datetimestamp
The time in which the process is submitted.

start_datetimestamp
The time in which the process formally begins. There can be a delay between this time and the creation_date due to the queue in which all processes have to pass through to properly allocate resources.

completion_datetimestamp
The time in which the process is stopped. This attribute is updated regardless of a failure, cancellation, or when the upload completes. Use the step attribute to verify the process state.

is_failedboolean
Defines whether the process has failed.

is_cancelledboolean
Defines whether the process has been cancelled.

error_messagestring
If a failure occurs, the attribute records the cause of the error. The full set of errors is located here.

More attributes

stream.subtitle_languagesarray
The selected subtitle languages.

Preview a URL

The sniff endpoint allows for a URL's metadata to be previewed before starting a process. If possible, this is recommended for stream downloads for the best experience when choosing the specifc formats to pull. If the URL is not compatible with Sorcerer or is unavailable, an error message will be returned stating the specific reason.

POST/sniff

$ curl https://api.sorcerer.io/sniff \
-H "client_id: YOUR_CLIENT_ID" \
-H "client_secret: YOUR_CLIENT_SECRET" \
-H "Content-Type: application/json" \
-d '{"url":"https://wikipedia.org/fake.archive.tar.gz"}'

RESPONSE FOR FILES

{
"file_name" : "fake.archive.tar.gz" ,
"mime_type" : "application/x-gzip" ,
"size" : 347489780
"url" : "https://wikipedia.org/fake.archive.tar.gz"
}

RESPONSE FOR STREAM MANIFESTS

{
"audio_tracks" : 1 ,
"file_name" : "fake.stream.manifest.m3u8" ,
"formats" : [

{
"audio_channels" : null ,
"audio_codec" : null ,
"audio_sampling_rate" : null ,
"audio_tracks" : [ "2" ]
"bandwidth" : 5759626 ,
"content_type" : "video" ,
"drm_protected" : true ,
"drm_providers" : [ "Widevine", "Microsoft PlayReady" ]
"duration" : 418 ,
"estimated_size" : 598158758 ,
"frame_rate" : 29.97 ,
"has_audio" : true ,
"height" : 1080 ,
"id" : "105" ,
"language" : "en" ,
"mime_type" : "video/mp4" ,
"subtitle_tracks" : [ "15" ]
"video_codec" : "avc1.640028" ,
"video_tracks" : [ ]
"width" : 1920
},
{...},
{...}

] ,

"subtitle_tracks" : 1 ,
"url" : "https://wikipedia.org/fake.stream.manifest.m3u8" ,
"video_tracks" : 1
}

Parameters

urlREQUIRED

Returns

For files, the sniff endpoint returns the file size in bytes and MIME type of the requested URL. For streams, the below is returned.

audio_tracksinteger
The number of audio streams in the manifest.

file_namestring
The name of the file.

formatsarray
An array of stream objects.

subtitle_tracksinteger
The number of subtitle streams in the manifest.

urlstring
The requested URL.

video_tracksinteger
The number of video streams in the manifest.

The Stream Object

Many of the stream object properties will revert to null if not provided by the manifest.

audio_channelsinteger
The number of audio streams for this particular format.

audio_codecstring
The audio codec of the stream if it is available.

audio_sampling_ratestring
The audio sampling rate.

audio_tracksarray
Array of the compatible audio stream format IDs.

bandwidthinteger
The bitrate of the stream.

content_typestring
Either video, audio, or subtitle.

drm_protectedbool
True if the stream is encrypted. Important to note that Sorcerer cannot download encrypted streams and will return an error message if attempted.

drm_providersarray
If the format is DRM protected, this array will contain a list of the DRM providers that encrypt the stream.

durationinteger
The duration of the format if available.

estimated_sizeinteger
The estimated size of the stream in bytes. This value is approximated using other values provided by the stream.

frame_rateinteger
The frame rate of the format if it is a video streams.

has_audiobool
True, if audio metadata is provided for the stream. Important to note that this may be false even if a video stream has audio because that metadata may not have been provided in the manifest.

heightinteger
The height resolution if the format is a video stream and if the information is provided by the manifest.

idstring
The ID for the format. Use this value to create processes.

languagestring
The language of the format. Often provided for audio and subtitle formats.

mime_typestring
The MIME type of the stream.

subtitles_tracksarray
Array of the compatible subtitle stream format IDs.

video_codecstring
The video codec of the stream if it is available.

video_tracksarray
Array of the compatible video stream format IDs. Only filled for audio and subtitle streams.

widthinteger
The width resolution if the format is a video stream and if the information is provided by the manifest.

Retrieve an upload process

GET/process/:process_id

$ curl https://api.sorcerer.io/process/2022-07-16T16:06:35.560ZbySbGCJ0vT \
-H "client_id: YOUR_CLIENT_ID" \
-H "client_secret: YOUR_CLIENT_SECRET" \

Parameters

No parameters.

Returns

Returns an upload process object.

Cancel an upload process

To be able to cancel a process, the upload cannot be at more than 90% completion. This is to ensure that there is enough time to properly terminate all connections. Additionally, if a process is cancelled, the data transferred is added to your monthly data usage. In particular, cancelled stream downloads count both the components and muxed file when cancelled after the components have been fully downloaded.

POST/cancel/:process_id

$ curl https://api.sorcerer.io/cancel/2022-07-16T16:06:35.560ZbySbGCJ0vT \
-H "client_id: YOUR_CLIENT_ID" \
-H "client_secret: YOUR_CLIENT_SECRET" \

RESPONSE

{
"cancelled" : true ,
"process_id" : "2022-07-16T16:06:35.560ZbySbGCJ0vT"
}

Parameters

No parameters.

Returns

Returns a cancelled boolean that specifies if the cancellation was successful and the process ID.

Webhooks

Webhooks can be set up in your account dashboard and a condensed upload process object is returned when a webhook is sent. You can have a maximum of five webhooks.

RESPONSE

{
"type" : "process.succeeded" ,
"process_id" : "2022-07-16T16:06:35.560ZbySbGCJ0vT" ,
"size" : 347489780
"error_message" : null ,
"step" : 6
"file_key" : "new-wikipedia-file-name.gz"
}

Errors

HTTP Errors

Sorcerer uses HTTP response codes to indicate the success or failure of an API request. Codes in the 2xx range indicate success. Codes in the 4xx range indicate an error that failed given the information provided. Codes in the 5xx range indicate an error with Sorcerer's servers.

Error Status Codes
200 - The request has worked.
400 - The request was unparsable.
401 - The API key was missing or invalid.
404 - The requested resource does not exist.
429 - Too many requests were sent to the API. Please slow down your request rate.
5XX - Something failed on Sorcerer's backend.

Common process errors

The file download failed due to Sorcerer failing to link to the provided connection. This error occurs if the connection attached to process is deleted or if the credentials are no longer valid.
Sorcerer failed to initiate the upload process. Access to the bucket may have changed. This error occurs if Sorcerer cannot build the upload due to a connection issue.
The bucket experienced too many requests. This error occurs if Sorcerer makes too many requests to the connection and is throttled. To avoid this error, avoid creating too many processes too close to each other.
The connection provided does not exist. This error occurs if the provided connection is deleted after the process is created.
Sorcerer does not have authorization to access that page. If the requested url blocks access to the file, this error occurs.

Docs

Get Started

Create an upload process

Parameters

Returns

The upload process object

More attributes

Preview a URL

Parameters

Returns

The Stream Object

Retrieve an upload process

Parameters

Returns

Cancel an upload process

Parameters

Returns

Webhooks

Errors

HTTP Errors

Common process errors

Ready to get started?