Lately, I worked on implementing the API of the main Git providers in some projects. I realized that we often use these tools but don't fully explore the resources they offer. So, I decided to share some of the initial steps I worked on, in a concise manner, to explain the most important aspects. In this article, we will cover how to access GitHub, GitLab, and Bitbucket APIs, retrieve repositories, and open issues.
You'll see that it's really easy and fun! Let's go!
Structure
Setting things up
Before diving into the code, we need to set up a few things for each provider:
Github
To access GitHub APIs, we need a Personal Access Token. Follow these steps to create one:
- Click on your profile pic in the top right corner of the page
- Go to Settings,
- In the left menu, click on Developer Settings
- Select Personal access Tokens
- Select Generate new token and then Tokens (classic)
- Give your token a recognizable name and choose the scopes that better suites your needs For our example, we will select the repo scope ;)
GitLab
GitLab also requires a Personal Access Token. Here's how you can create one:.
- Click on your profile pic in the top right corner of the page.
- Click on **Edit profile
- Click on Access Token
- Name your token and give it the access you need (we'll go for Api, read_user, read_repository, and write_repository)
- Click on "Create personal access token"
Here you go; you also have a GitLab access token.
BitBucket
There are several ways to gain access to Bitbucket. We'll go with the one that provides the most complete and durable access to the data. To do so, we need to have a project on Bitbucket that we can use for granting our application the credentials. Once you created a project:
- Click on the engine icon on the top right corner of the page
- Click on Workspace settings
- On the left menu, select OAuth Consumer.
- Click on Add Consumer.
- Select a name and a callback URL (e.g.,
http://localhost:3000/bitbucket/get-token) - Check Account, Workspace membership, Projects, Repositories, and Issue permissions
- Save the consumer and obtain the key and secret.
Since in this tutorial, we are also going to cover issues handling, Bitbucket requires that we create an issue tracker
N.B.: You can make it on any repository you want to create issues into, not necessarily the one you got the keys from.
Here are the steps to activate the issue tracker:
- open a repo you want to track the issues for
- go to repository settings
- go to issue tracker
- set public or private issue tracker and save Great! Now you can open issues.
The code
Now that we're all set up, we can put our hands on code ;)
Basic server
Let's start by creating our project! We will use an express.js server to develop our API endpoints because it's fast and light to set up, but you can choose any language and framework of choice.
In your terminal, run:
npm init
npm install axios body-parser cors dotenv express
If you don't have nodemon installed on your machine, install it by running:
npm install -g nodemon
Create an index.js file that will contain our server code:
//We import the basic libraries
const express = require('express');
const cors = require('cors');
const bodyParser = require('body-parser')
const axios = require('axios').default;
//then we import the .env file to use its variables
require('dotenv').config()
const app = express();
const port = 3000;
// setup the express server to work with API calls
app.use(bodyParser.json());
app.use(cors());
app.use(express.urlencoded({extended: true}));
//Now we are ready to start the server and make it listen to our request
app.listen(port,()=>{
console.log(`App listening on port ${port}`)
})
Let's start our server:
nodemon index.js
If everything is fine, you should see this message:
[nodemon] 2.0.22
[nodemon] to restart at any time, enter `rs`
[nodemon] watching path(s): *.*
[nodemon] watching extensions: js,mjs,json
[nodemon] starting `node index.js`
App listening on port 3000
BitBucket Authentication
Going in order, while GitHub and GitLab give you a bearer token that you can use for all of your requests, Bitbucket requires that you go with oAuth2 Authentication. This means that we need the user to authenticate on their website and authorize our application to use their API with the user's data. If the user authorizes our application, a code will be returned that will allow us to require the Bearer Token to make our requests.
This token lasts only 2 hours, so the request will also return a refresh token that we will need to receive a new token in the future.
It's longer than the other two but also pretty secure.
So let's authenticate.
First, let's paste the previously created key and secret in our .env file:
BITBUCKET_KEY=PUBLIC_KEY
BITBUCKET_SECRET=SECRET_KEY
Then let's begin our authentication process:
In our index.js file, let's create a first route that calls Bitbucket authorization page:
app.get('/bitbucket/auth', async (req, res) => {
const url = `https://bitbucket.org/site/oauth2/authorize?client_id=${process.env.BITBUCKET_KEY}&response_type=code`
res.redirect(url);
});
Then, supposing we gave our OAuth consumer the address http://localhost:3000/bitbucket/get-token, we can add the following route:
app.get('/bitbucket/get-token',async (req, res)=>{
const code = req.query.code;
try {
exec(`curl -X POST -u ${process.env.BITBUCKET_KEY}:${process.env.BITBUCKET_SECRET} https://bitbucket.org/site/oauth2/access_token -d grant_type=authorization_code -d code=${code}`, (error, stdout, stderr) => {
if(stdout){
const data = JSON.parse(stdout);
const bitbucketAccessData = {
token : data.access_token,
refresh_token : data.refresh_token,
}
let fileData = fs.readFileSync('./tokens.json');
fileData = JSON.parse(fileData)
fileData.tokens.bitbucket = bitbucketAccessData;
fileData = JSON.stringify(fileData);
fs.writeFileSync('./tokens.json', fileData, (err)=>{
console.error(err)
})
return res.json(data);
}
});
} catch (error) {
console.error(error);
return res.json(error)
}
})
Let's break this up.
The first route we wrote redirected us to Bitbucket authorization page.
Once we confirm on that page, we will be redirected to the URL we chose on the creation of our authConsumer. In this case, I decided http://localhost:3000 /bitbucket/get-token.
The redirect will occur with a query parameter named code that we must use to perform the following call.
For the following call, we now have the code in our query, bitbucket key, and secret in our env. We can now do a CURL call that Bitbucket needs for this authentication step.
From the data returned, we must store two pieces of information: the access_token and the refresh_token. The first will be used similarly to GitHub and GitLab Bearer Token. But it has a lifetime of only 2 hours. So we need the refresh token to ask for a new token, and when the time is over.
We will cover this topic later, anyway.
A glance at our repos
Now that we are all set up with our providers, we can start to gather information from our providers.
The first thing that may come to mind is to see which repositories are available on each platform. This way, we will be able to list them all, no matter which platforms they are, and we will be able to see them all in one place.
Let's first define the basis of our endpoints for our three providers:
const gitHubBasis = "https://api.github.com";
const gitLabBasis = "https://gitlab.com/api/v4";
const bitbucketBasis = "https://api.bitbucket.org/2.0"
Now let's see how to get our repositories finally: let's start with GitHub:
app.get('/git/repositories', async (req,res)=>{
const repoUrl = gitHubBasis + '/user/repos?per_page=10';
axios.get(repoUrl,{
headers:{
'Authorization': 'Bearer '+process.env.GITHUB_TOKEN,
//Be careful to the following two parameters, they NEED to be specified like this to work.
//N.B. The X-GitHub-Api-Version parameter might change depending on when you're reading the article, be sure to check the last version og Github Api
'X-GitHub-Api-Version':'2022-11-28',
'Accept': 'application/vnd.github+json'
}
}).then((response)=>{
res.json(response.data);
}).catch((err)=>{
console.log(err)
res.send(err);
})
})
Gitlab is pretty similar, as you can see
app.get('/gitlab/repositories', async (req,res)=>{
const repoUrl = gitLabBasis + '/projects?owned=1&per_page=10';
axios.get(repoUrl,{
headers:{
'Authorization': 'Bearer '+process.env.GITLAB_TOKEN,
'Accept': 'application/json'
}
}).then((response)=>{
res.json(response.data);
}).catch((err)=>{
console.log(err)
res.send(err);
})
})
As in previous steps, Bitbucket asks you to make more steps to reach your target! You have to choose a workspace from which you get the repositories. If you already know the workspace, good for you ;) But still, here's how you can get all the available workspaces for the logged user.
N.B. In this example, for shortness' sake, I store my tokens in a JSON file to make it easy and quick to call them. For a real-world application, find a way to store and read the token that suits your use case.
app.get('/bitbucket/workspaces', async (req,res)=>{
const wsUrl = bitbucketBasis + '/user/permissions/workspaces';
let fileData = fs.readFileSync('./tokens.json');
fileData = JSON.parse(fileData);
const token = fileData.tokens. Bitbucket.token;
axios.get(wsUrl,{
headers:{
'Authorization': 'Bearer '+token,
'Accept': 'application/json'
}
}).then((response) => {
res.json(response.data)
}).catch((error) => {
console.log(error);
res.send(error)
})
})
//now, using the workspace slug, we can access its repos :D
app.get('/bitbucket/repositories', async (req,res)=>{
const repoUrl = bitbucketBasis + '/repositories/'+req.query.ws;
let fileData = fs.readFileSync('./tokens.json');
fileData = JSON.parse(fileData);
const token = fileData.tokens. Bitbucket.token;
axios.get(repoUrl,{
headers:{
'Authorization': 'Bearer '+token,
'Accept': 'application/json'
}
}).then((response) => {
res.json(response.data)
}).catch((error) => {
console.log(error);
res.send(error)
})
})
Whatever the case, calling one of these endpoints will now receive a JSON with the list of your repositories! Nice, isn't it? :D
Now let's Publish something on them.
Opening an Issue
Once we have our repositories, we now want to handle issues for those repositories. A place to open an issue directly from where we are, independently from the provider the repository is hosted on, and also providing answers could be helpful. So wait no more; let's open our first issue.
Let's start trying opening an issue on the repository linked to this article, which is on GitHub:
app.post('/github/issue', (req, res) => {
const { owner, repo, issueTitle, issueText } = req.body;
const createIssueUrl = gitHubBasis + `/repos/${owner}/${repo}/issues`;
axios(createIssueUrl,{
method:'POST',
headers: {
'Accept': 'application/vnd.github+json',
'X-GitHub-Api-Version': '2022-11-28',
'Authorization': 'Bearer '+process.env.GITHUB_TOKEN,
},
data:{
owner: owner, //The repo owner username
repo: repo, // the repository name
title: issueTitle, // The title you want to give to the issue
body: issueText // and the issue content
}
}).then((response) => {
res.json(response.data)
}).catch((error) =>
console.error(error)
res.send(error)
})
})
The response we will receive will contain some API URLs, as well as a URL parameter with the address of the just-opened issue; here's an example:
{
"url": "https://api.github.com/repos/Giuliano1993/git-providers-article/issues/3",
"repository_url": "https://api.github.com/repos/Giuliano1993/git-providers-article",
"labels_url": "https://api.github.com/repos/Giuliano1993/git-providers-article/issues/3/labels{/name}",
"comments_url": "https://api.github.com/repos/Giuliano1993/git-providers-article/issues/3/comments",
"events_url": "https://api.github.com/repos/Giuliano1993/git-providers-article/issues/3/events",
"html_url": "https://github.com/Giuliano1993/git-providers-article/issues/3",
...
}
Next, we will add the route to comment on opened issues.
app.post('/github/issue/:id', (req, res) => {
const id = req.params.id;
const { owner, repo, commentText } = req.body;
const commentIssueUrl = gitHubBasis + `/repos/${owner}/${repo}/issues/${id}/comments`;
axios(commentIssueUrl,{
method:'POST',
headers: {
'Accept': 'application/vnd.github+json',
'X-GitHub-Api-Version': '2022-11-28',
'Authorization': 'Bearer '+process.env.GITHUB_TOKEN,
},
data:{
owner: owner,
repo: repo,
body: commentText,
issue_number: id
}
}).then((response) => {
res.json(response.data)
}).catch((error) => {
res.send(error)
})
})
The request is the same as opening an issue, slightly changing the endpoint and the issue_number parameter. Pretty straightforward, isn't it? Now let's do the same with our other two providers.
Getting the job done on GitLab is similar. Maybe slightly easier. On the other hand, you need to have your Project ID to perform your request: you can get it when making your repository list request, anyway ;)
Here's the code:
app.post('/gitlab/issue', (req, res) => {
const { issueTitle, issueDescription, repoId } = req.body;
const createIssueUrl = gitLabBasis + `/projects/${repoId}/issues`;
axios(createIssueUrl,{
method: 'post',
headers:{
'Authorization': 'Bearer '+process.env.GITLAB_TOKEN,
'Accept': 'application/json',
'Content-Type': 'application/json'
},
data:{
title: issueTitle,
description: issueDescription
}
}).then((response)=>{
res.json(response.data)
}).catch((error)=>{
res.send(error)
})
})
app.post('/gitlab/issue/:id/comment', (req, res) => {
const issueId = req.params.id;
const { commentBody, repoId } = req.body;
const createIssueUrl = gitLabBasis + `/projects/${repoId}/issues/${issueId}/notes`;
axios(createIssueUrl,{
method: 'post',
headers:{
'Authorization': 'Bearer '+process.env.GITLAB_TOKEN,
'Accept': 'application/json',
'Content-Type': 'application/json'
},
data:{
body: commentBody,
}
}).then((response)=>{
res.json(response.data)
}).catch((error)=>{
res.send(error)
})
})
As you can see, it is pretty similar to the GitHub way ;)
In this case, also Bitbucket doesn't differ much. Just remember 2 things:
- You need to pick not only the repository but also the workspace
- You must have the issue tracker active on the repository you want to open the issue. Otherwise, you will receive an
Error 404message. If you skipped that part earlier, you could find how to activate the Issue Tracker here
app.post('/bitbucket/issue',(req, res)=>{
const {workspace, repo, issueTitle, issueContent} = req.body;
const createIssueUrl = bitbucketBasis + `/repositories/${workspace}/${repo}/issues`
let fileData = fs.readFileSync('./tokens.json');
fileData = JSON.parse(fileData);
const token = fileData.tokens. Bitbucket.token;
axios(createIssueUrl,{
method:'POST',
headers:{
'Authorization': 'Bearer '+token,
'Accept': 'application/json'
},
data:{
title: issueTitle,
content: {
// Content accepts 3 types of content, that will display differently, based on where the issue is shown.
//raw, markup, html
// you can use all three or just one as I do here.
// markup and html gives a better organized issue and could be a good idea to include them
raw: issueContent
}
}
}).then((response) => {
res.json(response.data)
}).catch((error) => {
console.log(error);
res.send(error)
})
})
app.post('/bitbucket/issue/:id/comment',(req, res)=>{
const issueId = req.params.id;
const {workspace, repo, commentContent} = req.body;
const createIssueUrl = bitbucketBasis + `/repositories/${workspace}/${repo}/issues/${issueId}/comments/`
let fileData = fs.readFileSync('./tokens.json');
fileData = JSON.parse(fileData);
const token = fileData.tokens. Bitbucket.token;
axios(createIssueUrl,{
method:'POST',
headers:{
'Authorization': 'Bearer '+token,
'Accept': 'application/json'
},
data:{
content: {
raw: commentContent
}
}
}).then((response) => {
res.json(response.data)
}).catch((error) => {
console.log(error);
res.send(error)
})
})
And we can now create and comment on our issues also on Bitbucket! Not bad, isn't it?
Keep Trying things on
These were some elementary examples of API requests you can do to our beloved Git platforms. Requests may change slightly, and the code could get more complicated. Still, the point is: from this starting point, authentication and basic requests on some useful endpoints, it gets easier to navigate their APIs and getting confident with them: from here is up to you to make things more complex and fun.
You can find the repository with the entire code here
If this article was helpful and you would like me to dive deeper into this or other subjects, feel free to reach out in the comments or on Twitter @gosty93
I'll be happy for any feedback and ideas.
Happy Coding | _ 0
Top comments (0)