Understand RESTful API and Tests in One Post

RESTful APIs (Representational State Transfer APIs) are an architectural style for networked applications on the web, designed to take advantage of existing protocols. They are used to set rules that allow services to communicate over the Internet in a simple and standardized way.

In 2000, Roy Fielding introduced Representational State Transfer (REST) as a conceptual framework for developing web services. REST is a design paradigm for constructing networked applications that utilize hypermedia. While REST can operate over various protocols, it is typically implemented via HTTP for web services. This discussion will concentrate on the principles of crafting REST APIs within the context of HTTP.

REST is an architectural style defined by six constraints which, when applied as a whole, emphasizes scalability of component interactions, generality of interfaces, independent deployment of components, and intermediary components to reduce latency, enforce security, and encapsulate legacy systems.

Principles of RESTful APIs:

Uniform Interface: The uniform interface simplifies and decouples the architecture, which enables each part to evolve independently. The four guiding principles of a uniform interface in REST are:

Resource Identification in Requests: Each resource is identified by a stable URI, which ensures that each request from the client to the server targets a specific resource. For instance, a client might use the URI /users/123 to fetch information about user 123.
Resource Manipulation through Representations: When a client holds a representation of a resource, including any metadata attached, it has enough information to modify or delete the resource on the server, provided it has permission to do so.
Self-descriptive Messages: Each message includes enough information to describe how to process the message. Responses must therefore explicitly indicate their cacheability and, if applicable, carry their full or partial representation.
Hypermedia as the Engine of Application State (HATEOAS): Clients interact with the application entirely through hypermedia provided dynamically by server responses. The presence of hyperlinks within responses allows the client to discover actions and resources, navigating the full scope of the application without prior knowledge of its structure.

Statelessness: Statelessness means that no client context is stored on the server between requests. Each request from the client to the server must contain all the information the server needs to fulfill the request (authentication and authorization, if necessary). The server never relies on information from a previous request. This not only scales well but also simplifies the server design because it does not need to track or manage state.

Layered System: A REST API is designed to allow for a layered system architecture where the client cannot ordinarily tell whether it is connected directly to the end server or an intermediary along the way. This encourages the use of intermediaries like proxies and gateways, which can improve system scalability and manage security policies. However, the client’s request should eventually reach the end server that holds the resources, and the response should be the same as if there was no intermediary.

Cacheability: Caching in REST is applied to responses, allowing clients or intermediaries to store cacheable responses. This feature aims to reduce the need to repeat the retrieval of unchanged resources, thus enhancing efficiency and scalability. The server’s response must explicitly state whether it can be cached and for how long. This helps clients avoid the overhead of fetching the same data multiple times and reduces the load on the network and servers.

Code on Demand (optional): This is an optional constraint of REST and allows the server to extend the functionality of a client by transferring executable code. For example, when a web application sends JavaScript to the client in response to a particular request, the client can execute this script for client-side validation or to add new features. It’s the only optional constraint of REST architecture and can be used to reduce the number of features initially built into the client.

Benefits of RESTful APIs

Scalability:

Stateless interactions reduce server load.
Efficient caching minimizes data transfers.
Horizontal scaling is facilitated by statelessness.

Flexibility and Evolvability:

Decoupled client-server architecture allows for independent evolution.
Changes in server technology do not affect client applications.
Layered system design allows for updates without service disruption.

Technology Independence:

RESTful APIs can be implemented in any programming language.
Client and server can be updated or changed without affecting each other.
The API’s contract is preserved across various technology stacks.

HTTP Methods:

HTTP methods are a set of request methods that indicate the desired action to be performed on a given resource. Each of these methods implements a different semantic, but some common features are shared by a group of them. Here are the most commonly used HTTP methods:

GET: Requests a representation of the specified resource. GET requests should only retrieve data and have no other effect.
HEAD: Asks for a response identical to a GET request, but without the response body. This is useful for retrieving meta-information written in response headers, without having to transport the entire content.
POST: Used to submit an entity to the specified resource, often causing a change in state or side effects on the server.
PUT: Replaces all current representations of the target resource with the request payload.
DELETE: Removes the specified resource.
PATCH: Used to apply partial modifications to a resource.
OPTIONS: Describes the communication options for the target resource.
CONNECT: Establishes a tunnel to the server identified by the target resource.
TRACE: Performs a message loop-back test along the path to the target resource.

Each method tells the server what action to perform and can also inform the client about what actions are possible. For instance, whether a resource allows for deleting or only for retrieving. The design of these methods is such that they are idempotent (calling the same method multiple times will produce the same result) with the exception of POST, which is not idempotent.

HTTP Request:
A RESTful API request is made up of:

Endpoint: URL where the API can be accessed by the client application.
Method: The HTTP method, as mentioned above.
Headers: Provide information about the request or the server, like content type or authentication tokens.
Body: The data sent by the client to the server (not used with GET method).

HTTP Response:

(from: https://stackoverflow.com/questions/61061582/rest-api-hateoas-should-api-response-have-identifiers-as-hard-coded-or-as-plac )

A RESTful API response contains:

Status Code: Indicating success or failure of the request.
Headers: Similar to request headers, provide information about the response.
Body: The data being sent back to the client.

HTTP Error Codes:

They are part of the response message that the server returns when receiving an HTTP request. These codes are standardized by HTTP protocol and are categorized as follows:

1xx: Informational – Request received, continuing process.
2xx: Success – The action was successfully received, understood, and accepted.
3xx: Redirection – Further action needs to be taken in order to complete the request.
4xx: Client Error – The request contains bad syntax or cannot be fulfilled.
5xx: Server Error – The server failed to fulfill an apparently valid request.

Common HTTP status codes include:

200 OK: The request has succeeded.
201 Created: The request has been fulfilled and resulted in a new resource being created.
400 Bad Request: The server cannot or will not process the request due to something that is perceived to be a client error.
401 Unauthorized: The request has not been applied because it lacks valid authentication credentials for the target resource.
403 Forbidden: The server understood the request but refuses to authorize it.
404 Not Found: The requested resource was not found.
500 Internal Server Error: The server has encountered a situation it doesn’t know how to handle.

A RESTful API is thus a way of allowing communication between a web-based client and server that speaks HTTP. It’s an approach used in web development to create networked applications that are scalable and flexible.

Media Type

Media types, also known as Internet Media Types or MIME types, are standardized identifiers used to indicate the nature and format of a document, file, or assortment of bytes. They are important in REST APIs because they define the format of the data being transferred between a client and a server.

Here are some common media types used in REST APIs:

application/json: Represents JavaScript Object Notation (JSON) which is a common format for sending and receiving structured information.
application/xml: For XML data formats.
application/x-www-form-urlencoded: The default media type for submitting form data to the server.
multipart/form-data: Used when a form includes any <input type="file"> elements, for uploading files.
text/html: Standard media type for HTML documents.
text/plain: Media type for plain text without any formatting.
application/octet-stream: Represents any kind of binary data.
application/pdf: For PDF documents.
image/png, image/jpeg, image/gif: For image files in PNG, JPEG, and GIF formats respectively.
application/vnd.api+json: JSON API media type, for APIs that follow the JSON API specification.
application/ld+json: For JSON linked data.
application/hal+json: Representing JSON Hypertext Application Language format.
application/problem+json: For API problem details.

If the server doesn’t support the media type, it should return HTTP status code 415 (Unsupported Media Type).

When designing a REST API, it’s crucial to specify the media type that an endpoint accepts and returns. This is usually done through the Content-Type and Accept HTTP headers in the request and response. The Content-Type header tells the server the format of the request body, and the Accept header tells the server what format the client expects in the response.

Use Filtering, Sorting, and Pagination

Filtering:

Suppose we have an API for a book store, and we want to filter books based on genre and author.

API Endpoint without filtering:

GET /api/books

API Endpoint with filtering:

GET /api/books?genre=fiction&author=John+Doe

In this example, the API filters the books that are only in the ‘fiction’ genre and written by ‘John Doe’.

Sorting:

Now, if we want to sort these books by publish date in a descending order.

API Endpoint with sorting:

GET /api/books?genre=fiction&author=John+Doe&sort=-publishDate

The minus sign (-) indicates a descending order. If we omitted the minus, it would sort in ascending order by default.

Pagination:

To avoid returning too many books in one response, we paginate the results to show 10 books per page.

API Endpoint with pagination:

GET /api/books?genre=fiction&author=John+Doe&sort=-publishDate&page=1&limit=10

Here, page indicates the current page number, and limit specifies how many records to show per page.

Combined Example:

When you combine filtering, sorting, and pagination, the API provides a powerful and flexible way to query large datasets efficiently.

GET /api/books?genre=fiction&author=John+Doe&sort=-publishDate&page=2&limit=10

In this request, the client would receive the second page of ‘fiction’ books by ‘John Doe’, sorted by the most recent first, displaying 10 books per page.

Testing of RESTful APIs

Testing is a crucial part of the API development lifecycle, ensuring that the service is reliable, performant, and secure. Here’s an overview of the nine types of API testing you’ve mentioned, along with additional methods that may be relevant:

Validation Testing: This is about ensuring that the API meets the specifications and requirements. For REST APIs, validation testing would involve verifying the API endpoints, HTTP status codes, response payloads, and headers against the API specification (often defined in OpenAPI/Swagger).

Example: Checking that a GET /users/{id} endpoint correctly returns a 404 Not Found status code when the user does not exist.
Functional Testing: Functional testing looks at the business requirements and ensures that the API functions as expected. This includes testing individual endpoints for correct behavior, error handling, and ensuring that all API features fulfill the functional requirements.

Example: Ensuring that a POST /orders endpoint successfully creates an order and returns the order ID in the response.
UI Testing: Though REST APIs are backend services, they might be integrated with a frontend UI. UI Testing in the context of APIs would mean testing the API integration with the front-end application and ensuring that the API provides the correct data for the UI and handles UI requests as expected.

Example: Confirming that an e-commerce website correctly displays product details fetched from a GET /products/{id} endpoint.
Load Testing: Load testing assesses the API’s ability to handle large amounts of calls. This is where you simulate real-world usage and see if the API can maintain its performance under stress.

Example: Simulating thousands of simultaneous users to see how the API performs, identifying bottlenecks and performance degradation.
Runtime/Error Testing: This involves testing how the API behaves under various failure scenarios, such as network failures, server crashes, or resource limitations. It checks the API’s error messages, status codes, and error handling mechanisms.

Example: Triggering timeouts or manually shutting down a database to ensure the API returns an appropriate error message.
Security Testing: Security testing is about verifying that the API is protected against common security threats. This includes testing for vulnerabilities such as SQL injection, XSS, and ensuring authentication, authorization, encryption, and access control measures are working correctly.

Example: Testing API endpoints to ensure that a user cannot access another user’s data without proper authorization.
Penetration Testing: Penetration testing (or pen testing) is an authorized simulated attack on the API to identify vulnerabilities. Unlike automated security testing, pen testing is often manually performed by security experts.

Example: Attempting to exploit known security issues, such as outdated libraries or weak encryption, to gain unauthorized access to sensitive data.
Fuzz Testing: Fuzz testing, or fuzzing, is about inputting massive amounts of random data, or “noise,” to the API to try and force it to fail or reveal hidden bugs.

Example: Automatically sending a wide range of unexpected data types and formats to every input point of the API to see if it can handle data sanitization.
Interoperability Testing: This tests the API’s ability to interact and function correctly with other APIs, systems, and infrastructures. It ensures compatibility and adherence to industry standards.

Example: Ensuring that an API follows the OAuth 2.0 specification correctly when interacting with an identity provider.

Other Related Testing Methods:

Integration Testing: Verifying that different modules or services used by the API work together correctly. For example, checking if the API service correctly interacts with a payment processing service.
Contract Testing: Ensuring the API responses fit a previously agreed “contract” with the frontend or other consuming services. Tools like Pact can be used for this.
Performance Testing: More broad than load testing, it also includes testing the API’s response times, throughput, and resource usage under various conditions.
Compliance Testing: Checking if the API meets certain standards and regulations, which can be especially important in industries like finance and healthcare.
Smoke Testing: Doing a preliminary test to see if the API’s core functionality works before moving into more detailed testing.

For all these testing types, it’s essential to automate as much as possible to ensure consistency and efficiency. A robust CI/CD pipeline would typically include many of these testing phases to catch issues early and often. Remember that testing is not a one-time event but an ongoing process that should be integrated into the software development lifecycle (SDLC).

Ref: