What is a REST API? REST is an acronym for Representational State Transfer — an almost meaningless description of the most-used web service technology! A REST API is a way for two computer systems to communicate using the HTTP technologies found in web browsers and servers.

Sharing data between two or more systems has always been a fundamental requirement of software development. For example, consider buying motor insurance. Your insurer must obtain information about you and your vehicle so they request data from car registration authorities, credit agencies, banks, and other systems. All this happens transparently in real time to determine whether the insurer can offer a competitive policy.

A REST API Example

Open the following link in your browser to request a random computer question from the Open Trivia Database:

https://opentdb.com/api.php?amount=1&category=18

This is a public API implemented as RESTful web service (it follows REST conventions). Your browser will show a single JSON-formatted quiz question with answers, such as:

{ "response_code": 0, "results": [ { "category": "Science: Computers", "type": "multiple", "difficulty": "easy", "question": "What does GHz stand for?", "correct_answer": "Gigahertz", "incorrect_answers": [ "Gigahotz", "Gigahetz", "Gigahatz" ] } ]
}

You could request the same URL and get a response using any HTTP client, such as curl:

curl "https://opentdb.com/api.php?amount=1&category=18"

HTTP client libraries are available in all popular languages and runtimes including Fetch in JavaScript, Node.js, and Deno and file_get_contents() in PHP. A JSON response is machine-readable so it can be parsed and used before outputting HTML or other formats.

REST APIs and the Rest

Various data communication standards have evolved over the years. You may have encountered options including CORBA, SOAP, or XML-RPC. Most established strict messaging rules.

REST was defined in 2000 by Roy Fielding and is considerably simpler than the others. It’s not a standard but a set of recommendations and constraints for RESTful web services. These include:

  • Client-Server. SystemA makes an HTTP request to a URL hosted by SystemB, which returns a response.

    It’s identical to how a browser works. A browser makes a request for a specific URL. The request is routed to a web server which typically returns an HTML page. That page may contain references to images, style sheets, and JavaScript, which incur further requests and responses.

  • Stateless. REST is stateless: the client request should contain all the information necessary to respond.

    In other words, it should be possible to make two or more HTTP requests in any order, and the same responses will be received (… unless the API was designed to return random responses such as the quiz example above).

  • Cacheable. A response should be defined as cacheable or not.

    Caching improves performance because it’s not necessary to regenerate a response for the same URL. Private data specific to a certain user at a certain time would not normally be cached.

  • Layered. The requesting client need not know whether it’s communicating with the actual server, a proxy, or any other intermediary.

Creating a RESTful Web Service

A RESTful web service request contains:

  1. An Endpoint URL. An application implementing a RESTful API will define one or more URL endpoints with a domain, port, path, and/or query string — for example, https://mydomain/user/123?format=json.

  2. The HTTP method. Differing HTTP methods can be used on any endpoint which map to application create, read, update, and delete (CRUD) operations:

    HTTP methodCRUDAction
    GETreadreturns requested data
    POSTcreatecreates a new record
    PUT or PATCHupdateupdates an existing record
    DELETEdeletedeletes an existing record

    Examples:

    • a GET request to /user/ returns a list of registered users on a system
    • a POST request to /user/ creates a user with the ID 123 using the body data (see 4. below). The response returns the ID.
    • a PUT request to /user/123 updates user 123 with the body data (see 4. below)
    • a GET request to /user/123 returns the details of user 123
    • a DELETE request to /user/123 deletes user 123

  3. HTTP headers. Information such as authentication tokens or cookies can be contained in the HTTP request header.

  4. Body Data. Data is normally transmitted in the HTTP body in an identical way to HTML <form> submissions or by sending a single JSON-encoded data string.

REST API Response

The response payload can be whatever is practical: data, HTML, an image, an audio file, and so on. Data responses are typically JSON-encoded, but XML, CSV, simple strings, or any other format can be used. You could allow the return format to be specified in the request — for example, /user/123?format=json or /user/123?format=xml.

An appropriate HTTP status code should also be set in the response header. 200 OK is used for successful requests, although 201 Created could also be returned when a record is created. Errors should return an appropriate code such as 400 Bad Request, 404 Not Found, 401 Unauthorized, and so on.

Other HTTP headers can be set including the Cache-Control or Expires directives to specify how long a response can be cached before it’s considered stale.

However, there are no strict rules. Endpoint URLs, HTTP methods, body data, and response types can be implemented as you like. For example, POST, PUT, and PATCH are often used interchangeably so any will create or update a record as necessary.

REST API “Hello World” Example

The following Node.js code creates a RESTful web service using the Express framework. A single /hello/ endpoint responds to HTTP GET requests.

Ensure you have Node.js installed, then create a new folder named restapi. Create a new package.json file within that folder with the following content:

{ "name": "restapi", "version": "1.0.0", "description": "REST test", "scripts": { "start": "node ./index.js" }, "dependencies": { "express": "4.18.1" }
}

Run npm install from the command line to fetch the dependencies, then create an index.js file with the following code:


'use strict'; const port = 8888, express = require('express'), app = express(); app.get('/hello/:name?', (req, res) => res.json( { message: `Hello ${req.params.name || 'world'}!` } )
); app.listen(port, () => console.log(`Server started on port ${port}`);
);

Launch the application from the command line using npm start and open http://localhost:8888/hello/ in a browser. The following JSON is displayed in response to the GET request:

{ "message": "Hello world!"
}

The API also allows a custom name, so http://localhost:8888/hello/everyone/ returns:

{ "message": "Hello everyone!"
}

Client-side REST Requests and CORS

Consider the following HTML page launched in a browser at the URL http://localhost:8888/:

<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>REST test</title>
</head>
<body>
<script>
fetch('http://localhost:8888/hello/') .then((response) => { return response.json(); }) .then((json) => { console.log(json); });
</script>
</body>
</html>

The fetch call makes the same API request and browser console shows Object { message: "Hello world!" } as you would expect.

However, presume your RESTful web service was now put live on the Web at the domain http://mydomain.com/hello/. The page JavaScript fetch() URL is changed accordingly, but opening http://localhost:8888/ in the browser now returns the console error Cross-Origin Request Blocked.

For security, browsers only permit client-side XMLHttpRequest and Fetch API calls to the same domain where the calling page is hosted.

Fortunately, Cross-origin Resource Sharing (CORS) allows us to circumvent that security restriction. Setting an Access-Control-Allow-Origin HTTP response header tells the browsers permit the request. It can be set to a specific domain or * for all domains (implemented by the Quiz API above).

The web service API code can be changed to allow access from any client-side script running on any domain:


app.get('/hello/:name?', (req, res) => res .append('Access-Control-Allow-Origin', '*') .json( { message: `Hello ${req.params.name || 'world'}!` } )
);

Alternatively, an Express.js middleware function could append the header to every endpoint request:


app.use((req, res, next) => { res.append('Access-Control-Allow-Origin', '*'); next();
});

Note that browsers make two requests to the REST API:

  1. An HTTP OPTIONS request to the same URL determine whether the Access-Control-Allow-Origin HTTP response header is valid

  2. The actual REST call

When your server receives an OPTIONS request method it can set the Access-Control-Allow-Origin HTTP response header return a dummy empty response to ensure work is not duplicated.

REST API Challenges

The success of REST owes much to its simplicity. Developers are free to implement RESTful APIs as they like but it can lead to further challenges.

Endpoint Consensus

Consider the following endpoints:

  • /user/123
  • /user/id/123
  • /user/?id=123

All are valid options to fetch data for user 123. The number of combinations increase further when you have more complex operations. For example, return ten users whose surnames start with ‘A’ and work for companyX starting at record 51 when ordered by date of birth in reverse chronological order.

Ultimately, it doesn’t matter how you format URLs, but consistency across your API is important. That can be a challenge on large code bases with many developers.

REST API Versioning

API changes are inevitable, but endpoint URLs should never be invalidated or they’ll break the applications which use them.

APIs are often versioned to avoid compatibility issues. For example, /2.0/user/123 supersedes /user/123. Both the new and old endpoint can remain active. Unfortunately, it then becomes necessary to maintain multiple historical APIs. Older versions can eventually be scrapped but the process requires careful planning.

REST API Authentication

The quiz API shown above is open: any system can fetch a joke without authorization. This is not viable for APIs which access private data or permit update and delete requests.

Client-side applications on the same domain as the RESTful API will send and receive cookies just like any other HTTP request. (Note that Fetch() in older browsers requires the credentials init option to be set.) An API request can therefore be validated to ensure a user is logged in and has appropriate rights.

Third-party applications must use alternative methods of authorization. Common authentication options include:

  • HTTP basic authentication. An HTTP Authorization header containing a base64-encoded username:password string is passed in the request header.

  • API keys. A third-party application is granted permission to use an API by issuing a key which may have specific rights or be restricted to a particular domain. The key is passed in every request in the HTTP header or on the querystring.

  • OAuth. A token is obtained before any request can be made by sending a client ID and possibly a client secret to an OAuth server. The OAuth token is then sent with each API request until it expires.

  • JSON Web Tokens (JWT). Digitally signed authentication tokens are securely transmitted in both the request and response header. JWTs allow the server to encode access rights so calls to a database or other authorization system is not necessary.

API authentication will vary depending on the use context:

  • In some cases, a third-party application is treated like any other another logged-in user with specific rights and permissions. For example, a map API could return directions between two points to a calling application. It must confirm the application is a valid client but doesn’t need to check user credentials.

  • In other cases, the third-party application is requesting data private to an individual user such as email content. The REST API must identify the user and their rights, but it may not care which application is calling the API.

REST API Security

A RESTful API provides another route to access and manipulate your application. Even if it’s not a high-profile hacking target, a badly behaved client could send thousands of requests every second and crash your server.

Security is beyond the scope of this article, but common best practices include:

  • use HTTPS
  • use a robust authentication method
  • use CORS to limit client-side calls to specific domains
  • provide minimum functionality — that is, don’t create DELETE options which are not required
  • validate all endpoint URLs and body data
  • avoid exposing API tokens in client-side JavaScript
  • block access from unknown domains or IP addresses
  • block unexpectedly large payloads
  • consider rate limiting — that is, requests using the same API token or IP address are limited to N per minute
  • respond with an appropriate HTTP status code and caching header
  • log requests and investigate failures

Multiple Requests and Unnecessary Data

RESTful APIs are limited by their implementation. A response may contain more data than you need, or further requests are necessary to access all data.

Consider a RESTful API which provides access to author and book data. To show data for the top 10 selling books, the client could:

  • Request the first 10 /book/ details ordered by number of sales (top seller first). The response contains a list of books with each author ID.

  • Make up to 10 /author/{id} requests to fetch each author’s details.

This is known as the N+1 problem; N API requests must be made for each result in the parent request.

If this is a common use case, the RESTful API could be changed so that every returned book contained the full author details such as their name, age, country, biography, and so on. It could even provide full details of their other books — although this could considerably increase the response payload!

To avoid unnecessarily large responses, the API could be adjusted so author details are optional — for example, ?author_details=full. The number of options API authors need to cater for can become bewildering.

Does GraphQL Fix REST APIs?

The REST conundrums led Facebook to create GraphQL — a web service query language. Think of it as SQL for web services: a single request defines what data you need and how you want it returned.

GraphQL addresses some of the challenges posed by RESTful APIs although it introduces others. For example, it becomes difficult to cache GraphQL responses.

Your clients are unlikely to have problems comparable to Facebook, so it may be worth considering GraphQL once a RESTful API evolves beyond its practical limits.

There are numerous tools to help with RESTful API development in all languages. Notable options include:

  • Swagger: a variety of tools to help design, document, mock, test, and monitor REST APIs

  • Postman: a RESTful API testing application

  • Hoppscotch: an open-source, web-based alternative to Postman

There are also plenty of public REST APIs catering for jokes, currency conversion, geocoding, government data, and every topic you can think of. Many are free, although some require you to sign up for an API key or use other authentication methods. Categorized lists include:

Try consuming some RESTful APIs in your own projects before implementing your own web services.

Similar Posts