Building an ecosystem of open source connectors
Over the last few months, we have released a lot of connectors for Microsoft PowerApps and Microsoft Flow taking the number of connectors available to over 250. These connectors drive growth and adoption of the platform by enabling our customers to get started quickly building their apps and flows. It is also worth noting that all these connectors are built by partners – both internal partners within Microsoft and external partners. We have a process to certify these connectors.
Very often though, we get requests and feedback from our customers and community around features and gaps in many of these connectors. These could be as simple as the connector not providing a certain field/property to complex new triggers and actions that they would like to see. These feedbacks are usually captured in our forum ideas, and then we go about triaging and fixing the connectors. Sometimes, based on support tickets, we end up updating the connectors. But, like any feature, it takes time and we are limited by resources. We often end up recommending our customers to build a custom connector or use the HTTP connector to call the underlying API directly.
Open source connectors
As the number of connectors grow, we would like to empower the wider community. With that, I would like to announce that we are starting a new journey to open-source our connectors. Through an open sourced connector strategy, we hope to get more community developers involved in maintaining a large set of high-quality connectors. You can see how these connectors are authored, and try them out yourself. And, you should be able to fork these connectors to augment any feature gaps and bug fixes. In return, developers can submit those changes back so that the larger community can benefit.
As with any other feature, we like to make small but concrete steps. You can find out open source connector repository on GitHub. Today, we have a small set of connectors, but we are fully committed to add more connectors in the coming months. I would also like to state that we are going through an open-source by default strategy. All new connectors will be put up for open sourcing by default. While the ultimate decision to open source a connector lies with the owner of the connector, we strongly encourage all our partners to open source their connectors.
The repository also provides sample connectors to help developers get started. This is something that we would like to see your contributions as well. If you have built interesting custom connectors, we would like you to share your custom connector as a sample. In addition, we have updated our documentation as well with some new topics.
There are 2 platform features that enable us to open source our connectors. It will also highlight some of the challenge we have around open sourcing some of the connectors we have today.
Power Platform Connectors CLI
The CLI (Command Line Interface) is a powerful tool that can be used to download, create, or update any custom connector. This tool allows developers to save their custom connectors onto a local file storage, and have it managed using a source code management system. This workflow is not available with the portal – which is great to get started, but not great for developers. The CLI tool also provides the developers access to other advanced features not available in the portal like connection parameters.
To get started with the Power platform connectors CLI, follow the instructions here. The CLI is a cross platform utility that is based on the Azure CLI. So, if you are familiar with that, you will find that it is straightforward to use. Essentially, you need to install the Python runtime, and then install the CLI from the PyPI:
pip install paconn
Once installed, you can login and then start using the CLI to download, create or update a custom connector.
paconn login paconn download paconn create paconn update
In future, we will be adding more functionality to do validation of connectors as well. Just like our open source philosophy, the CLI itself is open sourced. You will be able to see the source code in the same connector repository on GitHub.
Policy support for custom connectors
Building a custom connector for Microsoft PowerApps and Flow is very straightforward. You simply describe the underlying API operations in OpenAPI format – which can be done by using our tooling in the portal. However, REST APIs are primarily designed to be used by developers who have intimate knowledge of the APIs. This can present a problem for custom connectors where the share of the operations exposed by the connector is determined directly by what is provided at the underlying APIs. This is where policies come in.
Policies can be used to modify the behavior of connectors at runtime. For example, policies are used to enforce throttling limits on API calls, to route calls to different endpoints, etc. Policies are used extensively by many of the out-of-box connectors we have today. We often end up writing many custom policies to modify the API behavior so that the end user experience is great. The use of these custom policies is a problem for us – and our partners. These policies are not available for custom connectors. That effectively means that partners cannot update their connectors or test the changes themselves. It also prevents us from open sourcing the connectors as there is no way for anyone to test those connectors.
That is, until now.
We now make available template-based policies to custom connectors as well. Policies are available in the Definition section for your custom connector.
We currently support the following templates:
- Dynamic host url – This policy allows the API calls to be routed to a host endpoint based on a connection parameter. This is useful if your connector is targeting different API URL. For instance, the Azure Key Vault connector sample leverages this policy to set the host.
- Route requests to a fixed endpoint: This policy can be used to route a request to another sub-path. Consider using this policy when there are multiple
- Set HTTP header: This policy can be used to set a specific HTTP header.
- Set query string parameter: This policy can be used to set a specific query parameter.
We will be adding more policy templates in future as we get more requirements.
Finally, putting it together – an example
In this example, the CLI and the policies are used to create a connector for Azure Key Vault that can dynamically route to different Key Vault endpoints based on a connection parameter. We believe there are many more examples like that, and we would love to hear them.
This marks an important step towards our journey to build a vibrant developer ecosystem. We would love to hear your feedback. Feel free to use our community forum or send an email directly to us: paconnfb@microsoft.com. We appreciate all feedback!