Implementing Custom Connectors
Fusebit adds ConnectorConnector - A connector is the package from Fusebit that manages the relationship between one or more integrations and a specific service. One of the most common types of connector is an OAuth connector, which takes care of the OAuth negotiation between your customers and the service you're integrating, so that you don't have to!s (reference) at a regular clip, but occasionally there will be a Connector that isn't yet part of the officially supported package. This might be an internal Connector, pointing at your API endpoints, or it might be a Connector needed to support a specific SaaS peer that your customer needs.
After checking in with us to make sure it's not already on our roadmap or in private release, you can follow this document to create your own Connector.
Estimated level of effort: 1-3 days
Scenario: Integrate with NewCo
Congratulations on a new sale! A customer has eagerly signed up for your product - but before the ink is even dry, they're demanding that you integrate with their other favorite software company: NewCo. NewCo is a fairly new SaaS that you've never heard of before, but this deal is worth a lot and you eagerly promise that the integration will be complete right away!
Fortunately, you have Fusebit in your corner! Let's build a Connector for NewCo that will keep your customer happy, and your developers happier.
Setting Up the Environment
Before we get started, let's make sure that we have the right tools in place.
- Install the Fusebit CLI via
npm install -g @fusebit/cli
- Sign in to your Fusebit account, or create a new account, by running
fuse init
- Your favorite editor - we use Visual Studio Code with various Javascript extensions installed, but any editor will do (Benn uses vim, the poor fellow)
- (Optional) Add the Fusebit internal npm registry to your npm search path via
fuse npm login
Collecting Information
Like any good project, we'll start this one off by collecting a few important pieces of information to help speed the process along later:
- Naming the Connector - sometimes you can get away with
newco
, but some SaaS (looking at you, GitHub) require special designations. - Identify if there is a standard package published on npm that you would expect to use to invoke NewCo's API. If there isn't one, the next best option is to use our built in
ApiClient
to provide a friendly HTTP method-based interface, please reach out to us via the Intercom messenger on this page for help. - Find the OAuth documentation for NewCo. Hopefully they published an easy-to-find page that includes the necessary URLs, but sometimes that page is distressingly difficult to find. If NewCo doesn't support OAuth, let us know and we'll help you out with the details.
Starting the Project
What Are We Building?
Let's talk about what we're going to build, and what building a Connector actually means for Fusebit. Fusebit separates out the relationship with a service like NewCo into two different pieces: a Connector, and a Provider.
A Connector handles OAuth, credential revocation, security issues, and other related secret management tasks. It runs in a separate context with different permissions than the IntegrationIntegration - An integration is the place the you write your code to get things done! Written in NodeJS, an integration runs in Fusebit's secure and scalable environment to translate between the needs of your backend application and the remote service or services you're connecting to. where you and your coworkers will write the meat of the functionality.
The Provider, on the other hand, acts as the local-to-the-Integration half of the Connector pair. It takes the security credentials for specific requested IdentityIdentity - An identity is a unique relationship one of your customers has with a service. An identity can be used by multiple integrations to act on that service on behalf of your customer. objects and converts them into a convenient, already configured SDK object. When you use code in the Integration that looks like this...
const sdk = await integration.service.getSdk(ctx, connectorName, ctx.params.installId);
... you're actually leveraging the code in the Provider to request the authorization token - usually a JWT - from the remote Connector, and then provision that into a SDK.
Steps To Build
There's a handful of steps that we're going to follow here to build out the Connector and the Provider:
- Create the Connector project
- Create a Provider project
- Adding the SDK
- Publishing the Connector and Provider to your private Fusebit npm Registry
- Deploying a new Integration and Connector!
We'll go through each of those steps in detail.
Let's Build!
Creating the Project Directories
Fusebit is built around NodeJS, so let's start by creating two new projects for the Provider and the Connector. We'll call them newco-connector
and newco-provider
for now.
Using the Private Fusebit npm Registry
If you want to publish your packages to the public npmjs.org registry, great! Many other people will be sure to benefit, and it's a great way to promote your support for NewCo to other NewCo customers!
But sometimes, you want to start with keeping a package private until it meets your bar for releasing into the wild. That's okay, we totally understand! We do the same thing ourselves, which is why we have a private npm Registry baked into your Fusebit Account.
You'll need to configure it to use a scope that's unique to you,
@mycompany
is a good choice. You can set that scope by using the Fusebit CLI we installed earlier:fuse npm registry scope set @mycompany # Then log in again to make sure your ~/.npmrc file is up to date fuse npm login
Once done, make sure to edit the
package.json
for each of the two projects and specify the name correctly:
{ "name": "@mycompany/newco-connector" }
and
{ "name": "@mycompany/newco-provider" }
.This will allow for each of the two projects to use
npm publish
to be published into the private Fusebit npm Registry, and be available to any Integrations that are being deployed in your Account.
Each of the projects can be initialized using the standard npm init
command:
$ mkdir newco-connector newco-provider
$ cd newco-connector
$ npm init
...
package name: (newco-connector) newco-connector
...
$ cd ../newco-provider
$ npm init
...
package name: (newco-provider) newco-provider
...
Initializing TypeScript and npm
Here's the starter project files we use, including Typescript, and a basic starter index.ts
file:
import { Connector } from '@fusebit-int/framework';
import { OAuthConnector } from '@fusebit-int/oauth-connector';
const TOKEN_URL = 'https://newco.com/oauth/access_token';
const AUTHORIZATION_URL = 'https://newco.com/oauth';
const REVOCATION_URL = 'https://newco.com/oauth/revoke';
const SERVICE_NAME = 'NewCo';
class ServiceConnector extends OAuthConnector {
protected addUrlConfigurationAdjustment(): Connector.Types.Handler {
return this.adjustUrlConfiguration(TOKEN_URL, AUTHORIZATION_URL, SERVICE_NAME.toLowerCase());
}
}
const connector = new ServiceConnector();
export default connector;
export { ServiceConnector };
{
"name": "newco-connector",
"version": "1.0.0",
"main": "libc/index.js",
"files": [
"libc/**/*.js",
"libc/**/*.d.ts",
"libc/**/*.json"
],
"scripts": {
"build": "tsc -b --pretty",
"lint:check": "eslint . --ext .ts --color",
"lint:fix": "eslint . --ext .ts --color --fix"
},
"dependencies": {
"@fusebit-int/oauth-connector": ">=7.24.1",
"superagent": "6.1.0"
},
"devDependencies": {
"@fusebit-int/framework": ">=7.24.1",
"@types/superagent": "^4.1.12",
"@typescript-eslint/eslint-plugin": "^4.31.0",
"@typescript-eslint/parser": "^4.31.0",
"eslint": "^7.32.0",
"eslint-config-prettier": "^8.3.0",
"eslint-plugin-import": "^2.24.2",
"eslint-plugin-security": "^1.4.0",
"typescript": "^4.4.3"
},
"peerDependencies": {
"@fusebit-int/framework": "*"
}
}
{
"compilerOptions": {
"target": "es2017",
"module": "CommonJS",
"declaration": true,
"declarationMap": true,
"sourceMap": true,
"strict": true,
"composite": true,
"esModuleInterop": true,
"moduleResolution": "node",
"resolveJsonModule": true,
"jsx": "react",
"forceConsistentCasingInFileNames": true,
"rootDir": "src",
"outDir": "libc"
},
"include": ["src", "src/**/*.json"],
"exclude": ["node_modules"]
}
And here is the Providers starting files:
import { Internal } from '@fusebit-int/framework';
import { Client } from '@newco/sdk';
type FusebitNewCoClient = Client & { fusebit?: any };
export default class NewCoProvider extends Internal.Provider.Activator<FusebitNewCoClient> {
/*
* This function will create an authorized wrapper of the NewCo SDK client.
*/
public async instantiate(
ctx: Internal.Types.Context,
lookupKey: string): Promise<FusebitNewCoClient>
{
const credentials = await this.requestConnectorToken({ ctx, lookupKey });
const client: FusebitNewCoClient = new Client({ auth: credentials.access_token });
return client;
}
}
{
"name": "newco-provider",
"version": "1.0.0",
"main": "libc/index.js",
"files": [
"libc/*.js",
"libc/*.d.ts"
],
"scripts": {
"build": "tsc -b --pretty",
"lint:check": "eslint . --ext .ts --color",
"lint:fix": "eslint . --ext .ts --color --fix",
},
"peerDependencies": {
"@fusebit-int/framework": "*"
},
"devDependencies": {
"@fusebit-int/framework": ">=7.24.1",
"@types/node": "^16.9.2",
"@types/request": "^2.48.7",
"@typescript-eslint/eslint-plugin": "^4.31.0",
"@typescript-eslint/parser": "^4.31.0",
"eslint": "^7.32.0",
"eslint-config-prettier": "^8.3.0",
"eslint-plugin-import": "^2.24.2",
"eslint-plugin-security": "^1.4.0",
"typescript": "^4.4.3"
},
"dependencies": {
"@newco/sdk": "1.0.0"
}
}
{
"compilerOptions": {
"target": "es2017",
"module": "CommonJS",
"declaration": true,
"declarationMap": true,
"sourceMap": true,
"strict": true,
"composite": true,
"esModuleInterop": true,
"moduleResolution": "node",
"resolveJsonModule": true,
"jsx": "react",
"forceConsistentCasingInFileNames": true
"rootDir": "src",
"outDir": "libc"
},
"include": ["src", "src/**/*.json"],
"exclude": ["node_modules"]
}
Use
peerDependencies
for@fusebit-int/framework
Most of the
package.json
andtsconfig.json
files are pretty boilerplate - there shouldn't be anything surprising to a Javascript or TypeScript developer there. There is one piece that is worth highlighting in thepackage.json
files however:"peerDependencies": { "@fusebit-int/framework": "*" }
The
peerDependencies
section ofpackage.json
is not commonly used, but in this case it is necessary to avoid proliferation of the@fusebit-int/framework
package, which is provided as a default part of the Connector and the Provider. Strange bugs can occur when this is not in thepeerDependencies
section!
Let's take a look at each of the index.ts
files...
newco-connector/src/index.ts
The newco-connector
package should export an instance of an object derived from the @fusebit-int/framework/Connector
class. This conforms to the specification that the system expects - and, incidentally, is almost identical to the contract that the Integration
object provides in the integration.js
you might have already written.
We've provided an OAuthConnector
object that does all of the nitty gritty mechanics of a "normal" OAuth handshake, including token rotation and various other capabilities. If the implementation that NewCo provides is not conventional, please let us know so we can help!
Let's start by importing in the Connector
and OAuthConnector
classes:
import { Connector } from '@fusebit-int/framework';
import { OAuthConnector } from '@fusebit-int/oauth-connector';
With those two objects in hand, let's define some constants that we picked up from the documentation we identified earlier:
const TOKEN_URL = 'https://newco.com/oauth/access_token';
const AUTHORIZATION_URL = 'https://newco.com/oauth';
const REVOCATION_URL = 'https://newco.com/oauth/revoke';
const SERVICE_NAME = 'NewCo';
We declare these at the top of the file so that they are easily and clearly identified - more than a few bugs have been created because those URLs were slightly off! And, for good measure, we include the name of the service we are supporting: NewCo
.
class ServiceConnector extends OAuthConnector {
protected addUrlConfigurationAdjustment(): Connector.Types.Handler {
return this.adjustUrlConfiguration(TOKEN_URL, AUTHORIZATION_URL, SERVICE_NAME.toLowerCase());
}
}
Now we've done the hard part of creating a Connector - calling adjustUrlConfiguration
so that the configuration shows up properly in the management portal!
We also created a new class ServiceConnector
that derives from the OAuthConnector
base class. This lets us inherit a whole bunch of work necessary to support an OAuth protocol. If NewCo isn't OAuth compatible, then reach out via Intercom for more details on how best to interface with them. We support a variety of interfaces for Connectors, depending on the mechanisms, security, and privacy considerations that are important.
Once we've created the ServiceConnector
class, let's return a new instance of it (and the class object, for good measure!) from this module.
const connector = new ServiceConnector();
export default connector;
export { ServiceConnector };
newco-provider/src/index.ts
The Provider acts to exchange the credentials supplied by the Connector and turn them into a viable client that the end developer can use. Here at Fusebit we try to make sure that the client returned matches both the public documentation as well as any sources and other tutorials that might be out there for the public SDK already. This allows the end developer - yourself, or your coworkers - to rapidly identify the "what" that they're trying to do, since they no longer have to worry about the "how".
We start the Provider off by importing in the Internal
module from @fusebit-int/framework
, as well as the published SDK from NewCo in the form of @newco/sdk
. NewCo, happily for us, returns a Client
object that contains their SDK, so we can focus our efforts there.
import { Internal } from '@fusebit-int/framework';
import { Client } from '@newco/sdk';
With the Client in hand, we declare a utility type so that your editor will recognize the typings from TypeScript...
type FusebitNewCoClient = Client & { fusebit?: any };
... and then declare a new class NewCoProvider
.
export default class NewCoProvider extends Internal.Provider.Activator<FusebitNewCoClient> {
This derives from Internal.Provider.Activator
, which expects the implementation to provide an instantiate
function.
public async instantiate(
ctx: Internal.Types.Context,
lookupKey: string): Promise<FusebitNewCoClient>
{
const credentials = await this.requestConnectorToken({ ctx, lookupKey });
const client: FusebitNewCoClient = new Client({ auth: credentials.access_token });
return client;
}
}
The instantiate
function takes as its parameters a Fusebit context object - this is the same object that is supplied to the routes in your integration.js
handlers - as well as a lookupKey
. This lookupKey
is often, but not always, a Identity object id, and is used by the Connector (in code that's part of the OAuthConnector
base class) to find the matching OAuth credentials.
This request returns a credentials
object which includes a valid access_token
. Happily, the Client
from @newco/sdk
takes that access_token
as a constructor parameter, and we're then able to return the client. The client will then be presented back to the caller of the getSdk
, getSdkByTenant
, or similar functions for use in your Integration.
Using the New Connector!
At this point, you've successfully created a new Connector leveraging the NewCo published SDK and their OAuth endpoints. Let's make use of it!
Start by compiling the projects:
$ cd newco-connector && npm run build
$ cd ../newco-provider && npm run build
Once they finish compiling, you can publish them to npm - either the Fusebit Private Registry, or the public npm registry, using npm publish
.
If you publish to the public registry, you will have to increase the version number each time you need to publish a new version. The Fusebit Private Registry is less concerned about supply-chain attacks since each registry is private to a specific customer.
Now you can use the Provider by specifying that package in the Integration's component list and package.json
:
{
"id": "example-integration",
"tags": {},
"handler": "./integration",
"components": [
{
"name": "newcoConnector",
"entityId": "newco-example-connector",
"provider": "newco-provider",
"entityType": "connector"
}
],
"componentTags": {},
"configuration": {}
}
{
"scripts": {
"deploy": "fuse integration deploy example-newco-integration -d .",
"get": "fuse integration get example-newco-integration -d ."
},
"dependencies": {
"superagent": "*",
"@fusebit-int/framework": ">5.2.0",
"newco-provider": "*"
},
"files": [
"./integration.js"
]
}
const { Integration } = require('@fusebit-int/framework');
const integration = new Integration();
integration.router.get('/api/:tenantId/test', async (ctx) => {
const sdk = await integration.service.getSdk(ctx, 'newcoConnector', ctx.params.tenantId);
// Use the fully authorized and operational NewCo SDK
const thing = await sdk.getAThing();
ctx.body = thing;
});
module.exports = integration;
This will make the Provider available when getSdk
is called with newcoConnector
as the function's parameter, and the NewCoProvider
will use the Connector located at newco-example-connector
in Fusebit for the class's authorization needs.
On the Connector side, we can create a new Connector called newco-example-connector
that makes use of our newco-connector
package:
{
"id": "example-newco-connector",
"tags": {},
"handler": "newco-connector",
"configuration": {
"mode": {
"useProduction": true
},
"scope": "offline_access api",
"clientId": "eY...Fd",
"clientSecret": "Ey...fD"
}
}
{
"scripts": {
"deploy": "fuse connector deploy example-newco-connector -d .",
"get": "fuse connector get example-newco-connector -d ."
},
"dependencies": {
"@fusebit-int/oauth-connector": ">5.2.0",
"@fusebit-int/framework": ">5.2.0",
"newco-connector": "*"
}
}
The handler
field in the fusebit.json
refers to the package name that the Connector is published under - so if you're using the registry, make sure it includes the scope from the package name!
What's Next?
At this point, you've successfully created and, hopefully, executed your new Integration, Connector, and Provider! Congratulations! But wait, there's more!
Fusebit also supports:
- Webhooks
- CRON and Scheduled events
- Pagination and queueing
Reach out to us via Intercom if you'd like support in building any of these capabilities into your new Connector!
Updated 30 days ago