Service Virtualization for SendGrid APIs

Overcoming Testing Challenges in SOA

Service-oriented architecture (SOA) is a widely adopted architectural pattern in modern software systems. It emphasizes the use of loosely coupled services to deliver functionality and perform various tasks. However, when it comes to testing projects that integrate with external APIs, several challenges arise. In this article, we will explore the concept of service virtualization, a powerful technique that simplifies the testing of API integrations, and how Beeceptor can assist in this process through its record and mock feature.

For example,

• how do we test the failure of one of the downstream APIs, and see application behavior?
• how to generate a massive response from an external API and test the behavior of our application, does it break on rendering, does it show the loader icon, etc?
• degraded performance and higher latencies of downstream APIs.
• how to cut costs (by not invoking external APIs) during performance testing.

The Importance of Integration Testing

Integration testing is a crucial aspect of ensuring that customer-oriented use cases are not compromised. In an SOA environment, applications often interact with external APIs or services to deliver comprehensive functionality. As a QA professional, you may have access to the front-end services (such as websites and apps), but controlling the behavior of downstream services without developer assistance can be difficult. For instance, consider the scenario of placing an order and sending a transactional email, a common feature in many applications. To facilitate this functionality, specialized transactional email services like SendGrid are commonly employed. However, testing this integration presents various challenges:

• Verifying that the email has been successfully triggered at SendGrid.
• Validating recipient information, such as correct CC recipients.
• Simulating timeouts for SendGrid APIs to ensure proper handling of service downtime.
• Testing the application's response to changes in SendGrid API response contracts, such as sending the email to a scheduled queue instead of triggering it instantly.

To address these challenges, a commonly used approach is to create a stub or test-double version of the SendGrid APIs. This technique, known as service virtualization, enables the testing of corner cases and facilitates effective integration testing.

Introducing Service Virtualization

Service virtualization is an approach that involves creating a simulated version of an external service to facilitate testing. It allows developers and QA professionals to test services in isolation, without relying on the availability of the actual external service. This approach offers several benefits, such as reducing testing time and costs by eliminating the need to trigger actual service calls, which might incur fees.

In our scenario, let's consider a software stack that utilizes SendGrid's transactional email API. To test the integration of this system, we can leverage Beeceptor to proxy traffic to api.sendgrid.com and record it. This recorded data can then be used to create mocked responses that emulate SendGrid's behavior. By doing so, we can effectively create a test-double and eliminate the dependency on the actual SendGrid APIs.

SendGrid - The Transactional Email Service

If you are building your application using Node.js, here's a standard code snippet that uses SendGrid's NodeJs SDK to trigger a transactional email. A similar code can be written in Java using SendGrid's Java SDK as well.

const sendGrid = require('@sendgrid/mail')
sendGrid.setApiKey(process.env.SENDGRID_API_KEY || "SG.some-api-key")
const msg = {
    to: 'quexahuhaule-4950@yopmail.com',        // Change to your recipient
    from: 'no-reply@your-domain.com',           // Change to your verified sender
    subject: 'Sending with SendGrid is Fun',
    text: 'and easy to do anywhere, even with Node.js',
    html: 'and easy to do <strong>anywhere</strong>, even with Node.js',
}

sendGrid.send(msg)
    .then((response) => {
        console.log(response[0].statusCode)
        console.log(response[0].body)
    })
    .catch((error) => {
        console.error('Error in sending email', error.response.body);
    });

If you save this code in app.js file, here is a shell command to run:

%> node app.js

1. Setting up Beeceptor Endpoint

To begin, let's create a Beeceptor endpoint and configure it in proxy mode. This way, any request made to the Beeceptor endpoint will be routed to the actual SendGrid APIs at api.sendgrid.com.

Set Beeceptor to forward requests to SendGrid's API domain. Beeceptor gives a named endpoint/subdomain that can be used as a base URL in our application.

2. Updating the Code and Modifying the Base URL

The SendGrid SDK provides a method to update the API base URL. In our case, we will set https://sendgridtest.free.beeceptor.com as the base URL, which we obtained from the Beeceptor endpoint page. Note that this change will only apply when the SENDGRID_API_MOCK_DOMAIN environment variable is enabled.

// SENDGRID_API_MOCK_DOMAIN = 'https://sendgridtest.free.beeceptor.com'
// initialize SendGrid SDK and setup API keys
if(process.env.SENDGRID_API_MOCK_DOMAIN) {
    sendGrid.client.setDefaultRequest('baseUrl', process.env.SENDGRID_API_MOCK_DOMAIN);
}
// send email now

This conditional check allows the same code to run in both sandbox and production environments. In the sandbox environment, you should set the SENDGRID_API_MOCK_DOMAIN environment variable to the Beeceptor endpoint address, https://sendgridtest.free.beeceptor.com.

Java

You can follow a similar syntax for SendGrid's Java SDK. The spec uses the default base URL as api.sendgrid.com, which can be overridden as shown below.

SendGrid sendGrid = new SendGrid(System.getenv("SENDGRID_API_KEY"));
String mockDomain = System.getenv("SENDGRID_API_MOCK_DOMAIN");
if(mockDomain != null && !mockDomain.isEmpty()) {
    sendGrid.setHost(mockDomain); // Set https://sendgridtest.free.beeceptor.com here 
}
// send email now

Flow diagram

With the code configuration completed, the call-flow is now set up as follows:

3. Sending a Request and Recording HTTP Request

Now, let's run the code. The SendGrid SDK will invoke an API call to Beeceptor, which will in turn route the request to api.sendgrid.com. Meanwhile, you can review the request and response on the Beeceptor dashboard.

Shell command to run:

%> SENDGRID_API_MOCK_DOMAIN=https://sendgridtest.free.beeceptor.com node app.js

In the following example, I haven't set an API key, resulting in an authentication error. However, don't worry! We will magically transform this call into a success in the next section.

On running app.js, Beeceptor receives a request. It is a 401 as we didn't provide valid API key.

4. Setting Up a Mock Rule

According to the SendGrid documentation for the /v3/mail/send API, we require a 202 HTTP status code. Let's click the Create Mock button on the request, and provide the expected JSON response and HTTP status code. This change will be instantly deployed on the Beeceptor mock server.

Creating a mocking rule in Beeceptor to send 202 response with a delay of a second.

5. Unleashing the Power of Mocks

When we run the same code again, the application will receive a success code, assuming that the email has been triggered. In reality, the mocked response is generated from the Beeceptor endpoint, and the request never reached SendGrid.

Congratulations! We have successfully unlocked the power of service virtualization!

Beeceptor sends a success response for new requests as a mock rule matches. We can inspect the request payload.

You can edit the mocking rule and try out a few more scenarios:

• Send a 500 status code to simulate SendGrid's downtime.
• Introduce a delay of 10 seconds to simulate increased SendGrid latencies.
• Generate an error message as per SendGrid's API error documentation and observe how your application behaves.

Conclusion

In conclusion, this article has explained the true benefits of service virtualization in testing API integrations, with an example of SendGrid's transactional email service. Leveraging Beeceptor's record and mock feature, developers and QA professionals can efficiently build test doubles.

Let's embrace service virtualization with Beeceptor to identify potential issues early, deliver high-quality software, and build resilient applications!