The Future of Building APIs with Data Handler

6 min readApr 19

It is not a surprise that, API is one of the main parts of programming.

APIs are a crucial part of the software. How much descriptive they are, it will be easy to use them, also to implement them.

Let’s see how being descriptive changes APIs look

paths:
  /pet:
    post:
      tags:
        - pet
      summary: Add a new pet
      description: Add a new pet
      operationId: addPet
      requestBody:
        description: Create a new pet
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/Pet'
          application/xml:
            schema:
              $ref: '#/components/schemas/Pet'
          application/x-www-form-urlencoded:
            schema:
              $ref: '#/components/schemas/Pet'
        required: true
      responses:
        '200':
          description: Successful operation
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Pet'          
            application/xml:
              schema:
                $ref: '#/components/schemas/Pet'
        '405':
          description: Invalid input
      security:
        - petstore_auth:
            - write:pets
            - read:pets

    Pet:
      required:
        - name
      type: object
      properties:
        id:
          type: string
          format: uuid
          summary: Random uuid will be generated upon creation time
        name:
          type: string
          example: doggie
        tags:
          type: array
          items:
            type: string
        status:
          type: string
          description: pet status in the store
          enum:
            - available
            - pending
            - sold

From this file, we can understand how it will work, if we need such an API we just need to implement it. It is a straightforward process.

I think that all business logic, which API customers will know, must be somehow described in the API contract

Here, we get one question!

If we can somehow describe our APIs better, it will explain its business logic, and by just looking at our API contract, we can understand everything and implement it. Can we build automated software where it does implementation for us?

The answer is Yes and No

So, mostly, we can build software, which automatically implements our API from its declaration (e.g. swagger yaml file). But there are some technical limitations.

It is not possible to describe every complex business logic in API declaration
We cannot mix, imperative style coding with declarative APIs. It means that if we start to enrich API declaration and include logic definitions like functions, if, for, etc. API declaration will become very complex, It will be against our purpose.

So, that’s why our answer to the question, is yes and no.

We can build software to automate API implementation from its description. But its limits will be reached on complex/custom business logic

Why not keep this barrier? So we can have an automated application that will do the implementation for us, but also keep a door for custom implementation.

It was my idea. So I built Data Handler for that.

With Data Handler. You can define declarative APIs (in their own format) and it will build an implementation for you, automatically. But for the situations, where it is not possible to cover business logic in a declarative way, you have the ability to implement it by extensions (how it is named in Data Handler).

Let’s see an example.

Prerequisites:

You need to install the Data Handler server and its client tool (dhctl). Please follow this link for the installation

pet.yml

type: resource
name: pet
title: Pet
description: API for Pet
properties:
  - name: name # property name
    required: true
    type: STRING
    length: 255
    title: Name
    description: Name of Pet
  - name: description # property name
    type: STRING
    length: 255
    title: Description
    description: Name of Pet
  - name: tags # property name
    type: LIST
    subProperty:
      type: STRING
  - name: status # property name
    type: ENUM
    enumValues:
      - available
      - pending
      - sold

Now let’s call the `dhctl` tool (it is a CLI tool for “Data Handler”) to apply our yaml declaration

dhctl apply -f pet.yml

When you apply this yaml file, Data Handler will automatically create everything for you. It includes, it will create, a database table (by default data handler uses PostgreSQL as a database), and it will create Rest API, Swagger, Grpc API, etc. for you.

Let’s see

curl -X 'POST' \
  'http://localhost:9009/pet' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer <token>' \
  -H 'Content-Type: application/json' \
  -d '{
  "name": "string",
  "description": "string"
}'

What if, we want to set the description automatically based on the title if it is not defined?

Basically, we want following

If description is empty
  description = name + ' Pet'

As we see it is a custom logic, and we cannot define custom logics in a declarative easily. Either it will not be possible OR We will end up with very complicated declarative language. Like a mix of declarative and imperative syntax.

How Data Handler can help us?

Data Handler supports extensions, so you can easily extend Resource (in our case, Pet is a resource) operations. So, let’s do it!

We will use Nodejs for this example. But Data Handler can work on different languages also (for now, only Nodejs and Golang are natively supported)

First, let’s setup package.json

{
  "name": "store-app",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "start": "npx ts-node src/index.ts",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "data-handler-client": "1.0.10",
    "@types/node": "^18.15.11",
    "typescript": "^5.0.4"
  },
  "dependencies": {
    "axios": "^1.3.5"
  }
}

Now, let’s prepare our typescript schema for Pet Resource

dhctl generate -p=src --platform=nodejs

It will generate schema.ts for us with the following content

// auto generated by dhctl generate tool
import {Entity} from "data-handler-client";

export interface Pet extends Entity {
    id: string;
    name: string;
    description: string;
    tags: array;
    status: string;
    version: number;
}

Now, let’s run our code for changing the description upon creation time

import {Pet} from "./schema";
import {DhClient} from "data-handler-client";

const client = new DhClient({
    Addr: "127.0.0.1:9009", // data handler server address
    Insecure: true,
})

async function run() {
    await client.authenticateWithUsernameAndPassword("admin", "admin")
    const petRepo = client.newRepository<Pet>("default", "pet")
    
    // because we are running Data Handler inside docker, we need to set remoteUrl to the host machine: "http://host.docker.internal:17686"
    const extension = client.NewExtensionService("127.0.0.1", 17686, "http://host.docker.internal:17686") // which port we will run extension // which port we will run extension

    await extension.run()

    const petExtension = petRepo.extend(extension)

    petExtension.onCreate(async (pet) => {
        if (!pet.description) {
            pet.description = pet.name + ' Pet' 
        }

        return pet
    })
}

run()

Now if you create a pet without a “description”, it will add a “description” automatically for you.

Let’s see

curl -X 'POST' \
  'http://localhost:9009/pet' \
  -H 'accept: application/json' \
  -H 'Authorization: Bearer <token>' \
  -H 'Content-Type: application/json' \
  -d '{
  "name": "string"
}'

It will create the following Pet for us

{
  "description": "string Pet",
  "id": "8ac80eae-dea4-11ed-9751-0242ac110002",
  "name": "string",
  "version": 1
}

Great! We have customized our business logic by automatically implementing most of the logic.

What else?

By continuing this approach, we can enrich declarative functionality, we can have better support to handle most of the cases, easily. But we always need a door to have the ability for imperative codes and custom implementations.

Next, we can even integrate AI tools like Open API, and Chat GPT, so it can create a declarative config for us, It will be even easier to build application APIs with just a piece of words.

Let’s see.

I am asking Chat GPT following sentence

Prepare me a resource definition in Data Handler accepted format for following: 
We have a Pet, it has name, description, tags, status (one of pending, accepted, processed)

type: resource
name: "pet"
properties:
  - name: "name"
    type: "string"
    length: 255
    required: true
  - name: "description"
    type: "string"
    length: 255
  - name: "tags"
    type: "array"
    items:
      type: "string"
      length: 50
  - name: "status"
    type: "string"
    length: 50
    allowed_values:
      - "pending"
      - "accepted"
      - "processed"
    default: "pending"

Small mistakes, which can be improved.

Nowadays, AI tools are not better at writing whole applications. They do a lot of critical mistakes. But if we give them Declarative language (like I showed in Data Handler). AI will be much more efficient on it.

In the future, I am planning to integrate OpenAPI into Data Handler, so you can build your APIs by just describing them in your pure English language without any special syntax.

If you like my idea, please star my repository http://github.com/tislib/data-handler. It will be a motivation for me to continue my work on this.

Also, your feedback will be very helpful to me.

Feedbacks are welcome in any place including

https://github.com/tislib/data-handler/discussions

https://github.com/tislib/data-handler/issues

https://www.linkedin.com/in/talehibrahimli/