A startup founder reached out to me regarding possibly joining in a machine learning engineer role. As someone who has mostly worked with research in last three years, and not on engineering, it was both flattening and daunting to receive this invite. After initial discussion on my interests and career plans as well as an informal description of their product; I was offered to appear for an interview about a week later. Unfortunately, I realised that I could not join according to the timeline mentioned by the company so I politely declined their offer to interview. As I was very interested in solving the problem mentioned by them so I decided to attempt their hiring interview question on my own time.

The problem statement was to take a pre-trained machine learning model and turn it into an API. I chose the DistilBERT transformer model for text classification available from HuggingFace. The model receives natural language text broken down into tokens and classifies it as positive or negative in sentiment. The POST call would be taking a natural language input, our model would turn it into tokens, do the classification and finally return the classification confidence and class.

http POST http://127.0.0.1:8000/classify text="Pre-trained DistilBERT seems to work quite well!"

{
    "confidence": 0.9998160004615784,
    "probabilities": {
        "negative": 0.00018407008610665798,
        "positive": 0.9998160004615784
    },
    "sentiment": "positive"
}

While I followed a tutorial to build the API, I introduced sufficient novelty through using a different model and not using the exact libraries etc. I was able to get my API up and running on my local computer. I also tried to get it running remotely on Heroku but couldn’t get it to work. So I will work on getting Heroku deployment done the next time.

Key Learnings

• HuggingFace Transformers
• FastAPI

Next

I tried to put my api on a server using Heroku but could not get it to work. I suspect this is because the server downloads a pre-trained model and stores it in memory before replying to a POST calls. This is not possible since Heroku’s memory is ephemeral and does not allow downloading/storing files (from what I have read). Of course, I am not familiar with Heroku to comment on this in any detail. Next steps are deploy the API to Heroku, or any other cloud platform. I also plan to write unit tests using a python test framework.