0
votes

I'm attempting to translate a .zip file into a .svf manifest using Autodesk's Model Derivative API. I am able to successfully create a bucket, place the zip into the bucket using chunked uploading, and begin a translation job using the respective endpoints (all endpoints return 200). When I come to check the translation progress of the job, it stalls at "0% complete", and eventually returns a failure message of TranslationWorker-InternalFailure.

This is all done programmatically on a stack of Node.js & superagent to make HTTP requests. I am able to confirm that the file itself is valid, by manually translating it using Paw; and also by translating it with our old tool which we're replacing.

See below for the specific sections of my code.

Autodesk Translation Job (Endpoint)

translate: async function(accessToken, obj) {
    return await request.post('https://developer.api.autodesk.com/modelderivative/v2/designdata/job')
        .set("Authorization", "Bearer " + accessToken)
        .set("Content-Type", "application/json")
        .set('x-ads-force', true)
        .send({
            input: {
                urn: obj.base64Urn,
                compressedUrn: true,
                rootFilename: obj.parentFile
            },
            output: {
                formats: [
                    {
                        type: "thumbnail"
                    },
                    {
                        type: "svf",
                        views: ["2d", "3d"]
                    }
                ]
            }
        });
}

In the above code, the variables have the following values:

  • obj.base64Urn is dynamically converted from the URN provided by Autodesk via the following function:

        base64Urn: function() {
            // https://tools.ietf.org/html/rfc7515#appendix-C
            return this.getDataValue("urn") != null ? (new Buffer(this.getDataValue("urn"))).toString('base64').replace(/=+$/g, "").replace(/\+/g, "-").replace(/[\/]/g, "_") : null;
        }
    
  • obj.parentFile takes the form of "160728 Small Test Project.rvt"

Autodesk Translation Progress (Endpoint)

getTranslationProgressAndDerivatives: function(accessToken, obj) {
    return request.get('https://developer.api.autodesk.com/modelderivative/v2/designdata/' + obj.base64Urn + '/manifest')
        .set('Authorization', "Bearer " + accessToken);
}

Of course, the obj has to be placed on Autodesk's servers at some point. This is accomplished by this piece of code:

Autodesk Put Object into Bucket (Endpoint)

class HttpAutodeskPutObjectWriteStream extends stream.Writable {

    /**
     * Constructor sets all the properties for the instance.
     *
     * @param {string} accessToken - The OAuth2 access token needed for server authentication to engage with Autodesk's API.
     * @param {Obj} obj
     */
    constructor(accessToken, bucket) {
        super();
        this.accessToken = accessToken;
        this.obj = obj;
        this._bytesTransferred = 0;
        this._putHttpUrl = 'https://developer.api.autodesk.com/oss/v2/buckets/' + this.obj.name + '/objects/' + this.obj.name + '/resumable';
    }

    /**
     * Return a bytes transferred statement.
     *
     * @private
     *
     * @param chunk - The chunk currently being transferred.
     *
     * @returns {string}
     */
    _bytesTransferredStatement(chunk) {
        return `bytes ${this._bytesTransferred}-${this._bytesTransferred+chunk.byteLength-1}/${this.obj.zipFileSize}`;
    };

    /**
     * Writes data to the stream. Note the use of the serialize method in the request.
     *
     * @private
     *
     * @param chunk - The chunk currently being transferred.
     * @param encoding - The encoding of the chunk data.
     * @param callback - The function to be called on chunk completion (success or otherwise).
     *
     * @returns {Promise.<void>}
     */
    async _write(chunk, encoding, callback) {
        try {
            let stmt = this._bytesTransferredStatement(chunk);
            this._bytesTransferred += chunk.byteLength;

            let response = await request.put(this._putHttpUrl)
                .set("Authorization", "Bearer " + this.accessToken)
                .set("Session-Id", this.bucket.key)
                .set("Content-Length", chunk.byteLength)
                .set("Content-Range", stmt)
                .serialize(function(d) { return d; })
                .send(chunk.toString());

            if (response.status === 200) {
                this.urn = response.body.objectId;
            }
            callback(null);

        } catch (e) {
            callback(e);
        }
    };
}

Which is then invoked here:

put: function(accessToken, bucketEntity) {
    return new Promise(async (resolve, reject) => {
        const maximalChunkedTransferSize = 2*1024*1024; // 2MB (Minimum amount, Autodesk recommends 5MB).

        const objStorageLocation = (await config.get())[process.env.NODE_ENV].objStorageLocation;
        const pathToFile = path.join(__dirname, "../../", objStorageLocation, obj.name + ".zip");

        let readFileStream = fs.createReadStream(pathToFile, { highWaterMark: maximalChunkedTransferSize });

        let ws = new HttpAutodeskPutObjectWriteStream(accessToken, obj);
        readFileStream.pipe(ws);

        ws.on("finish", () => resolve(ws.urn));
        ws.on("error", () => reject());
    });
}

When the job fails...

Once it fails, this is the response I receive from the translation progress endpoint:

{
    "type": "manifest",
    "hasThumbnail": "false",
    "status": "failed",
    "progress": "complete",
    "region": "US",
    "urn": "<redacted>",
    "version": "1.0",
    "derivatives": [
        {
            "name": "LMV Bubble",
            "hasThumbnail": "false",
            "status": "failed",
            "progress": "complete",
            "messages": [
                {
                    "type": "error",
                    "message": "Translation failure",
                    "code": "TranslationWorker-InternalFailure"
                }
            ],
            "outputType": "svf"
        }
    ]
}

Am I doing anything obviously wrong here that would result in a 100% failure rate when attempting to translate files?

1

1 Answers

1
votes

And naturally, posting on StackOverflow makes you mentally review your code from a different perspective. The problem was in my uploading of the file, specifically, the last chained method:

let response = await request.put(this._putHttpUrl)
    .set("Authorization", "Bearer " + this.accessToken)
    .set("Session-Id", this.bucket.key)
    .set("Content-Length", chunk.byteLength)
    .set("Content-Range", stmt)
    .serialize(function(d) { return d; })
    .send(chunk.toString()); // HERE

It is incorrect to call .toString() on the chunk.