In March of 2024, an entrepreneur looking to develop an email analysis web app approached us to help create a backend service that used an AI tool to analyze emails.
For businesses and individuals, email security is a constant concern with the growing cases of phishing scams, and malware.
Our mission was to design a backend service that could efficiently analyze emails using the OpenAI SDK. We set out to create a system that could determine if an email was “safe” by checking for several key factors.
For example, an email would be considered safe if it came from a verified sender, had no suspicious links or attachments, is free of misleading content and excessive urgency, and avoided unsolicited offers or requests for sensitive information.
It took us only a month to develop this powerful web app. This is the story of how we turned their vision into reality, integrating OpenAI to create a powerful email analysis tool.
Process
Here’s how we developed and implemented the process of integrating OpenAI into our backend service for email analysis, which allowed for individual category ratings and the calculation of a final risk score:
1. Email Data Reception and Preparation
The first step involved setting up an endpoint in our backend application to receive raw email data from the client’s frontend application.
2. Data Cleaning
To ensure the integrity and usability of the data, we implemented a cleaning process to strip out all unnecessary HTML, JavaScript, and CSS tags (for templated emails). We created custom functions leveraging regular expressions in order to achieve this. For the email headers, we included various parameters such as “spf”, “dkim”, “ARC-Seal” to assess its authenticity.
3. Data Categorization
After cleaning, the data was divided into predefined categories: “email_header”, “email_content”, “event_info”, “attachments”, and “sender_info”. This was done in order to get back individual category ratings from OpenAI and calculate the final rating based on the weights of each category. Each category had different weights and it varied based on whether the email was from a contact or not (provided by the FE).
4. Interaction with OpenAI API
For each categorized data segment, we crafted specific prompts and sent these to the OpenAI API. Each request was tailored to evaluate the content of its respective category for risks or other relevant attributes. We used the chat completion model to analyze the cleaned data.
5. Calculating and Returning Final Data
After OpenAI performed the analysis, it returned a numerical rating and reasoning for each email category in a JSON format as it was instructed. We then calculated the final rating based on the weighted average of these individual ratings. Finally, both the final rating and the detailed assessments for each category were saved to the database and sent back to the client. If an email id already exists in the database, then the data is returned from the database, else the data will be cleaned and sent to OpenAI.
OpenAI Setup
OpenAI has a chat completion SDK available in various programming languages, designed to streamline the interaction between system and user prompts. In the system prompt, a detailed description is provided to guide the AI’s task: to receive categorized email data, analyze it, and determine a risk level on a scale from 0 to 5 (5 being the safest). The AI is instructed to accompany each risk level assessment with a reason, formatting the output in JSON for clarity and consistency. Below is a sample of the data returned from OpenAI based on our prompts.
{
"headers_rating": {
"value": "value",
"reason": "reason"
},
"content_rating": {
"value": "value",
"reason": "reason"
},
}
This structured format ensures that each aspect of the email is individually assessed and explained, providing comprehensive insights into potential risks associated with the email contents.
Fig: Prompting chat completion with cleaned and categorized email data
Limitations
OpenAI has imposed a few limitations in their SDKs that should be taken into consideration. The major ones to look out for are Tokens Per Minute (TPM) and Requests Per Minute (RPM).
Fig: Rate limits on various models
Technology used
The following tools and technologies were used for this project:
- Python
- FastAPI and Uvicorn
- OpenAI Chat Completion
Conclusion
In conclusion, this case study highlights a successful integration of OpenAI’s capabilities into a backend service for analyzing email data. By utilizing OpenAI’s chat completion SDK, we were able to efficiently categorize email content and evaluate risk levels, thus enhancing the security measures for email communication.
However, time was definitely a constraint for us. To meet the tight deadline, we relied on OpenAI’s pre-trained models. While this solution was effective, there is a possibility of greater potential with a custom-trained model.
Instead of giving the system level prompt on every request, a custom model could be created with these prompts and just the user level prompts could be provided in a request which would reduce the token usage.
This fine-tuning would not only save on resources but also further enhance the accuracy of our email analysis.
That’s it for today! Thank you for reading.
.……………….
Gurzu is a full-cycle software development company. Since 2014, we have helped world-class customers get to their markets quickly with high-quality products built with modern software technologies. Our team of experienced developers, designers, and test automation engineers can help to develop your next product.
Have a tech idea you want to turn into reality? Book a free consulting call or simply, leave us a text message.
