Integration Programming Model

In this section, you will learn about the different concepts important to writing your IntegrationIntegration - An integration is the place the you write your code to get things done! Written in NodeJS, an integration runs in Fusebit's secure and scalable environment to translate between the needs of your backend application and the remote service or services you're connecting to.. Integrations are based on Node.js and the full power of the npm ecosystem, which enables you to build some pretty sophisticated things into your Integration: data transformations, multi-system call orchestration, caching, and data cleanup among other things.

Every Integration is a Node.js module, making it easy to manage and distribute. At its most basic, an Integration looks like this:

const { Integration } = require('@fusebit-int/framework');
const integration = new Integration();
const router = integration.router;

router.get('/api/tenant/:tenantId/test', integration.middleware.authorizeUser('instance:get'), async (ctx) => {
  
  // Do integration work

  ctx.body = {text: 'Hello World!'};
});

module.exports = integration;
{
  "id": "sampleIntegration",
  "tags": {},
  "handler": "./integration",
  "configuration": {},
  "componentTags": {},
  "components": []
}
{
  "scripts": {
    "deploy": "fuse integration deploy sampleIntegration -d .",
    "get": "fuse integration get sampleIntegration -d ."
  },
  "dependencies": {
    "@fusebit-int/framework": "^3.0.0"
  },
  "files": [
    "./integration.js"
  ]
}

An Integration is an HTTP service, where you as the developer define the routes that make sense for your scenario. Normally, you would define routes that make it easy to invoke the necessary functionality from your own application. You control the route shape and payload, so tailor it for ease of use, and handle any integration complexity inside Fusebit.

The router property of the Integration object exposes a Koa-based router. Koa is a modern Node.js framework for building web applications and APIs, and you can reference its documentation whenever you're in doubt about how the router and related concepts work.

Fusebit Tenancy Model

One of the key elements of Fusebit is its multi-tenant architecture, which allows you as the developer to accommodate all your system's users with their own private integration credentials and configuration settings. For example, if you build an expense reporting SaaS, each seat within each company you sell into is likely tied to a different user with their own Slack identity. Each user within each company will need to be notified separately of events that pertain to them, and they may choose to configure the Slack integration differently. Fusebit makes it super easy to handle that complexity.

Fusebit uses the concept of a TenantTenant - A single user of an integration, usually corresponding to a user or account in your own system. to capture every individual user in your system that you would like to enable to install integrations. As you probably noted in the snippet above, you explicitly pass the tenantId when you invoke every Fusebit route, so we can look upon whose behalf to operate.

router.get('/api/tenant/:tenantId/test', integration.middleware.authorizeUser('instance:get'), async (ctx) => {
  // ... 
}

The tenantId is opaque in Fusebit. You can use any value in your own system to identify your users. So, think of it as a "foreign key." By default, it is passed as part of the route path, but you have the flexibility to pass it in any way you see fit.

Invoking Third-Party Systems

Every Fusebit Integration comes with an attached ConnectorConnector - A connector is the package from Fusebit that manages the relationship between one or more integrations and a specific service. One of the most common types of connector is an OAuth connector, which takes care of the OAuth negotiation between your customers and the service you're integrating, so that you don't have to! (or multiple) that enables it to talk to third-party systems such as Slack. The Connector performs many functions, but the key is storing and refreshing the credentials of all Tenants of that integration.

📘

The connector credentials are stored securely since they're encrypted at rest.

The way you access the third-party system for a Connector is through an authenticated SDK that Fusebit provides for you, for example:

router.post('/api/tenant/:tenantId/test', integration.middleware.authorizeUser('instance:get'), async (ctx) => {
  const slackClient = await integration.tenant
    .getSdkByTenant(ctx, 'slackConnector', ctx.params.tenantId);
  // Do something with slackClient and set ctx.body
});

You will notice that the tenantId is passed into the getSdkByTenant call. If you modify the way the tenantId is passed into your integration, you will need to supply it here.

The resulting slackClient object is the authenticated SDK you would use to Slack on behalf of your user. Fusebit exposes. Fusebit exposes the most popular npm package available for the third-party app (using the official one, when available), so you can count its whole community and documentation to get support.

📘

For Slack we expose the official @slack/web-api SDK.

In this simple example, we are using the postMessage method to send a "hello world" message.

router.post('/api/tenant/:tenantId/test', integration.middleware.authorizeUser('instance:get'), async (ctx) => {
  const slackClient = await integration.tenant
    .getSdkByTenant(ctx, 'slackConnector', ctx.params.tenantId);
  const result = await slackClient.chat.postMessage({
    text: 'Hello world from Fusebit!',
    channel: 'demo',
  });
  ctx.body = result;
});

Protecting your API

Usually, the routes you define in an integration require protection against unauthorized access.
As you may already notice, the mechanism to protect your routes it's through the authorizeUser middleware. This function restricts access to users authenticated in Fusebit with the specified permission.

If you need more sophisticated user authentication mechanisms, please reach out to us. We offer full support for various in-house and external authentication systems and can help you build the right solution for your platform.

router.post('/api/tenant/:tenantId/test', integration.middleware.authorizeUser('instance:get'), async (ctx) => {
   // Implement your code here
});

Storage

Our SDK also provides a secure and reliable storage mechanism. With this mechanism, you can write, read, update, and delete hierarchical data in a versioned fashion.

Write data

You can save any data in JSON format up to ~400Kb in size. You need to consider a key name representing a reference to your data that you will use in further operations like read, delete and update. Think about it like a file name where you're placing your data.

router.post('/api/tenant/:tenantId/favorite/:color', async (ctx) => {
  const result = await integration.storage.setData(ctx,
    `/${ctx.params.tenantId}/favorite`,
    { color: ctx.request.body.color }
  );

  ctx.body = result;
});

Read data

To read data, you need to reference the key name you've used for storing the data. You will get the result in JSON format.

router.get('/api/tenant/:tenantId/favorite', async (ctx) => {
  const result = await integration.storage.getData(ctx, `/${ctx.params.tenantId}/favorite`);

  ctx.body = `The users favorite color is ${result.data.color}`;
});

Update data with versioning

You need to get the stored data by referencing the key name you've used for it to update data. Then you apply the changes to returned data and call setData method with the updated values that include the versioning information from the original getData. This prevents conflicts when multiple writers may attempt to write at the same time.

A conflict will generate an exception to be handled by the caller.

router.put('/api/tenant/:tenantId/colors', async (ctx) => {
  const key = 'colors';
  const currentColors = await integration.storage.getData(ctx, key);
  const newColors = [...currentColors, 'purple'];
  const result = await integration.storage.setData(ctx, key, newColors);
  ctx.body = result;
});

Delete data

To delete data, reference the key name you've used for storing the data and call deleteData.

router.delete('/api/tenant/:tenantId/colors', async (ctx) => {
  const key = 'colors';
  const result = await integration.storage.deleteData(ctx, key);
  ctx.body = result;
});

Listing data

A listing operation query data stored in an artifact known as a BucketBucket - A container of storage objects, think about them like a directory that contain multiple files that store data. Buckets are collections of keys where you can store related data.

Before querying a bucket, first, you need to learn how to store data in one. In fact, you already used buckets in some of the previous examples. A few moments ago, you used the tenantId as the bucket name when you stored the favorite color of a tenant (/${ctx.params.tenantId}/favorite). However, you are not restricted to this use case. You can choose and use any arbitrary string as the bucket name.

Let's see an example:

router.post('/api/tenant/:tenantId/colors', async (ctx) => {
  // By convention we use / symbol to represent a bucket, but you can use any name you want.
  const bucketName = '/my-bucket/';
  const key = 'colors';
  const data = ['green', 'blue'];
  const result = await integration.storage.setData(ctx, `${bucketName}${key}`, data);
  ctx.body = result;
});

Let's add more items to the /my-bucket/ bucket.

router.post('/api/tenant/:tenantId/shapes', async (ctx) => {
  const bucketName = '/my-bucket/';
  const key = 'shapes';
  const data = ['circle', 'triangle'];
  const result = await integration.storage.setData(ctx, `${bucketName}${key}`, data);
  ctx.body = result;
});

So far, you've added data under the shapes and colors keys to the /my-bucket/ bucket. To list these items, you can use the listData method, as shown here:

router.get('/api/tenant/:tenantId/my-bucket', async (ctx) => {
  const bucketName = '/my-bucket/';
  const result = await integration.storage.listData(ctx, bucketName);
  ctx.body = result;
});

By default, the SDK will return up to 5 keys. In the previous examples, you've added only have two: shapes and colors. But if you add more, you might want to specify the count argument when calling listData to fit your needs.

router.get('/api/tenant/:tenantId/my-bucket', async (ctx) => {
  const bucketName = '/my-bucket/';
  // restrict the maximum returned items from the query parameter named count
  const count = ctx.request.query.count;
  const result = await integration.storage.listData(ctx, bucketName, { count });
  ctx.body = result;
});

If the amount of keys in a bucket is higher than the specified count, you will receive a next object in the result. You can use this object to paginate through the next pages, as show here:

router.get('/api/tenant/:tenantId/my-bucket', async (ctx) => {
  const bucketName = '/my-bucket/';

  // It's up to you how to provide the filtering variables, the most common way is via query string
  // Restrict the maximum returned items by using the count query parameter
  const = ctx.request.query.count;

  // Paginate the results by passing the next query parameter
  const next = ctx.request.query.next;
  const result = await integration.storage.listData(ctx,bucketName,{ count, next });
  ctx.body = result;
});

Scheduling Jobs

Once you've modified your Integration to your needs, you can also configure it to run on a pre-set scheduled cadence using the Fusebit Cron Scheduler.

To do this, you need to make changes to two files:

  1. fusebit.json where you specify the schedule to run the integration on
  2. integration.js where you specify which endpoint you want to for the scheduled job

Specify the Schedule

The fusebit.json file contains the configuration logic for your specific integration. In this file, you will want to update three items:

  • Cron: The cron schedule you want to run the integration on
  • Timezone: The timezone to sync the cron scheduler to
  • Endpoint: The specific endpoint you want to run (You will define this in the integrations.js file)
{
  "schedule": [
    {
      "cron": "* * * * *",
      "timezone": "America/Toronto",
      "endpoint": "/poll-hubspot"
    }
  ]
}

📘

Please note that we currently only accept one schedule per integration at the moment. Reach out to us if you would like to increase this limit!

Specify the endpoint

In the integration.js file, you will define the endpoint, that you want the CRON job to pick up and run as per your scheduled configuration.

router.cron('/poll-hubspot', async (ctx) => {
  // do the thing.
});

Once you've completed this and deployed your integration to Fusebit, this endpoint will start getting called on the defined cadence.