Make your business data and functionality available for internal departments, external business, users and systems. Every company, every department is working on an API strategy and implementation to fulfill the needs of their users, build a digital platform for their business.
There are many ways to build an API platform. This article on ProgrammableWeb describes different (maturity) levels of API design.
- Level 1 (Unintentional): Publishing of unstructured or poorly structured data sets as PDFs or CSV files.
- Level 2 (Automation): Publishing APIs using tools that auto-generate “APIs” directly from an existing database.
- Level 3 (Self Design): Manual CRUD-centric API design commonly used in “RESTful” API design.
- Level 4 (Activity-focused): Activity-focused design happens when the team bases API design on identifying target users of the API, researching these users and understanding their most common activities.
With these levels in mind you can look at different design and implementations flavors for an API platform using Azure and external resources.
Level 2, Automation (let’s forget level 1).
The Microsoft platform doesn’t offer any service for automation on the generation of API’s directly from a database. There are several tools and Microsoft investigated several implementations which come close to generated endpoints.
In 2010 Microsoft worked on a framework that exposed SQL Server tables as OData endpoints, but this project was discontinued. There are some third party tool vendors who make tools that make this possible. For example, SQL Server Cloud Driver from CData has a server side component on top of SQL server for this.
Interesting to look at the Ecosystem of OData producers on Odata.org which lists services that expose their data using the OData protocol.
Azure Table Storage is one of them.
Windows Azure Table provides scalable, available, and durable structured storage in the form of tables exposed as OData endpoints.
A no code solution for providing business data to end-users, can be fulfilled by filling a Table Storage with data and make the Table Storage Endpoints available for consumers. Read for endpoint details: https://msdn.microsoft.com/en-us/library/azure/dd179423.aspx
The data exposed via the OData Azure Table Storage endpoints can come from different resources, from on premises SQL Server databases and from internal and external systems. This data can be captured, aggregated and transformed to the table storage via Azure Data Factory. Azure Data Factory is a cloud data integration service to compose data storage, movement, and processing services into data pipelines. A cross on-premises and cloud data sources and SaaS. For details: https://azure.microsoft.com/en-us/documentation/articles/data-factory-introduction/ .
The gateway with the on premises datacenter the Data Factory makes it easy to connect systems through the firewall.
Exposing data via a Table Storage and the Data Factory, makes it a codeless implementation of a read only API which exposes business data to the consumers.
The authentication to the Azure Table Storage OData endpoints require the users to know the Azure Storage keys or from the Azure portal a SAS token can be created.
By configuring a new Data Pipeline in the Data Factory and API Management Endpoints in a fast way new data can be exposed.
This implementation is a little bit more than the Level 2 ‘Automation’ implementation as described in the Programmable web article, because the data exposed can be configured via the data pipelines.
- Fast implementation
- Read only
- Consumer must have business knowledge.
- Technical knowledge needed a consumer side.
Level 3 Self Designed CRUD API’s.
OData services are CRUD based query able REST services. There are two ways to create an OData REST service with the .NET framework. WCF Data Services and Web API with the OData controller. A good comparison between the two can be found in this post: WCF Data Services and Web API with OData; choices, choices. The generic point, there is a need for development which takes time but also brings some additional flexibility.
Making these data endpoints available to consumers can be done via Azure API Management. The connection through the firewall can vary. One option for an on premises connection is via an Azure Service Bus Relay. An Azure Service Bus Relay publishes APIs to external and internal consumers, securely and protecting them from abuse and overuse. A gateway proxy can be made to gain insights into usage and health.
A nice example is : Azure API Management + Azure Service Bus + Minecraft
Making endpoints available to consumers on a structured way can be accomplished by using Azure API Management, read: https://azure.microsoft.com/en-us/services/api-management/.
- On premises management of services.
- CRUD only.
- Tight connection to the database model.
Another implementation of the ‘Level 3 CRUD API’s’ can be accomplished with an Azure API App. The implementation can make use of a VPN (or Azure Express Route) configuration for the connection with the on-premises databases and expose the OData endpoints via an API App. Implemented with Web API OData Controller, ASP.NET OData. See: Create an OData v4 Endpoint Using ASP.NET Web API 2.2.
- Easy to create CRUD services.
- In control about the data to expose.
- VPN connectivity necessary.
- CRUD only.
- Tight connection to the database model.
Another implementation can be done with an Azure SQL Database connected with an Azure API App. The Azure SQL Database can be filled, synchronized with Azure SQL Data Sync.
Azure SQL Data Sync is a cloud-based data synchronization service built on the Microsoft Sync Framework technologies. It provides bi-directional data synchronization and data management capabilities allowing data to be easily shared across SQL Databases across multiple data centers. The current release is in Preview.
- Controlled load on local database
- Easy to build services
- No VPN connection
A note on the Level 3 CRUD scenario’s, there are more configurations and combinations possible.
Level 4 (Activity-focused). APIs Are Products.
An API platform must be developed with the same mindset as a product is developed. It must bring business functionality, benefits to the consumer. A well thought-out strategy and implementation is needed to bring this value to the consumers (read: The open banking era).
The implementation will take some design effort with strong principles. Make real REST services (REpresentational State Transfer) with the principles of HATEOAS (Hypermedia As The Engine Of Application State).
For some good information read this API design guidance from Microsoft Patterns and Practices. And more reading on:
- A real digital platform
- Business value for consumers
- Big effort
- Long term commitment
Note: there are many more configurations and combinations possible, also between the different levels. In a Digital Platform a combination will be needed, next to a combination of protocols, authentication and formats, this is only the beginning.