googleCloud2

Google cloud functions and cron jobs

Abstract

This article is more about building a real time, fast, scalable, time-based and event driven application with the help of app engine cron services and cloud functions. We will explore how cron jobs and cloud functions can be so helpful sometimes.

Introduction

What are cron jobs ?

Cron jobs are time-based jobs which are programmed to run on a specific time interval. App Engine cron service automatically triggers the cron jobs that operate at a regular interval or defined time. App Engine cron service will invoke a URL by using a HTTP get request.

What are Google Cloud Functions ?

Google cloud functions are server less compute solution which are used to create event-driven applications. It is a joint product of Google cloud platform and firebase team. Firebase is a development platform for mobile and web applications, founded in 2011 and acquired by Google in 2014.

Following things can be done with cloud functions.

  • 1.  can be used for notification when a certain event triggers.
  • 2.  can be used for Real time Database which syncs data across all the clients in real time.
  • 3.  can be used to perform intensive task at cloud instead of in your own application.It’s kind of help in computation which makes application run more faster.
  • 4.  can be integrated with third party APIs.

To explore more about cron jobs and cloud functions, we will create a real world problem. Then, we will see how cron jobs and cloud functions are so helpful sometimes to solve these kind of problems, which developers often face while creating a web or mobile application.

My Meeting Example:

My_Meeting (SQL Table)

SQL Table

SQL table contains the data of the meetings which has to be held between a student and a mentor.The meetings will be done remotely through video chat at the scheduled time.

N number of meetings (video conference) can be held between students and mentors on scheduled time in a single day. For example, there are only five meetings in the My_Meeting SQL table.

What are the challenges ?

From above table, it is clear that a meeting can happen anytime in the day. So on the basis of meeting start time (see the SQL table ), we have to create a video chat room (third party API would be used for video chat) for the scheduled meeting (neither before start time, nor after start time).

Traditional approach to solve the problem

Normally, the traditional approach is to write the business logic in our backend to find out the list of meeting which has to go on and has to create the video chat rooms. Keep in mind that checking every minute to find out the list of the meeting which has to be done, will keep our backend always busy. By the way, it is a highly intensive task as a thread will check endlessly that “Is current time equal to any meeting start time (see the SQL table)”, “if yes , create a video chat room “, “if no , wait for next minute to check it again”. Our backend will be always busy in this computational task .Remember meetings table can contain thousands of rows(meetings) . So it is not suggested to do this job on server side to achieve better performance. Hence, we will go for cloud functions and Google cron jobs.

Another approach to solve the problem

As we have decided that we will not go with traditional approach on server to find out the list of the meetings, which are meant to be held right now. Let’s go with app engine cron service and cloud functions to arrange and schedule the meetings. It will make our app run fast , scalable and reduces the computational task of our server. Now let’s take a dive in the code and see how helpful cloud compute solutions can be sometimes for our applications and developers.

see the following diagram which explains the cloud compute solution approach .

appEngine

Prerequisite:

  • 1. A Google account
  • 2. Create a Developers Console project.
  • 3. Installation (or check that you have previously installed)
    • a. Node and npm
    • b. Google cloud SDK
    • c. Enable the pub/sub API
    • d. Enable project billing

Now we are ready to code and deploy our first cron job which will be invoked by app engine every minute (in this example).

Do following steps (command-line)

  • 1. mkdir appengine
  • 2. cd appengine
  • 3. type nul > package.json
  • 4. type nul > app.js
  • 5. type nul > app.yaml
  • 6. type nul > cron.yaml

Now we have the following files in our current working directory.

Files

Now let’s feed some code to the files .

Step 1.

Open the package.json file and include the following code in the file and do command npm install.


{
"name": "functions- cron -appengine",
"version": "1.0.0",
"description": "AppEngine component for functions- cron ",
"main": "app.js",
"engines": {
"node": "8.x.x"
},
"scripts": {
"deploy": "gcloud app deploy --quiet app.yaml cron.yaml",
"start": "node app.js"
},
"license": "Apache-2.0",
"dependencies": {
"@google-cloud/pubsub": "^0.20.1",
"express": "^4.16.3"
}

 

Step 2.

Open the app.js file and include the following code in the file .


const express = require('express');
const PubSub = require('@google-cloud/pubsub');
const pubsubClient = new PubSub({
projectId:’your google developer console projectId ’);
const app = express();
app.get('/publish/:topic', async (req, res) =>{
const topic = req.params['topic'];
try {
await pubsubClient.topic(topic)
.publisher()
.publish(Buffer.from('test'));
res.status(200).send('Published to ' + topic).end();
} catch (e) {
res.status(500).send('' + e).end();
}
});
const PORT = process.env.PORT || 6060;
app.listen(PORT, () => {
console.log(`App listening on port ${PORT}`);
console.log('Press Ctrl+C to quit.');
});

Step 3.

Open the app.yaml file and include the following code in the file .


runtime: nodejs8

instance_class: F1

automatic_scaling:

max_instances: 1

 

Step 4.

Open the cron.yaml file and include the following code in the file .

cron:

description: Push a "tick" onto pubsub every minute
url: /publish/min-tick
schedule: every 1 minutes

We are too close to deploy our first cron job. As you can see that cron.yaml contains a schedule
field. in this field we have to mention the time interval on which we want to run this cron job by app engine cron service . In our case it will execute the job on every minute.

Lets’s deploy the cron jobs on google cloud

Do the following steps in the same directory (command-line).

  • 1. gcloud auth login
  • 2. gcloud config set project your_google_console_projectid
  • 3. gcloud app create
  • 4. gcloud app deploy app.yaml cron.yaml

Congratulations, you have successfully deployed your first cron job. Now google app engine will execute this task on every minute (In this case ).

Now it’s time to deploy our firebase cloud function. So, without wasting any
time, let’s take another dive into the code.

Do the following steps (command-line).

    • 1. mkdir mycloudfunctions
    • 2. cd mycloudfunctions
    • 3. npm install -g firebase-tools
    • 4. firebase init

Firebase Init

Choose the 3 rd option which is firebase functions.

    • 5. Say yes , to install the dependencies ?

Your directory will look like this.

files2

  • 6. Cd functions
  • 7. npm install knex
  • 8. Open the index.js file and overwrite the file with the following code.


const functions = require('firebase-functions');
const knex = require('knex'); // or any other client
const dbConfig = {
client: 'your_db_client',
connection: {
host: 'your database address',
user: 'username',
password: 'password',
database: 'your_database_name'
}
}
let db = knex(dbConfig);
exports.minutely_create_room_job = functions.pubsub
.topic('min-tick')
.onPublish((message) => {
console.log("this job runs every minute by appengine service");
findMeetingsToBeStarted();
});
function findMeetingsToBeStarted(){
let date = new Date();
date.setSeconds(0, 0);
db.table('My_Meeting')
.where({ startTime: date.toISOString() })
.select()
.then(result => {
/*
to do

create here video chat room for meetings and update the other necessary parameters in the database.
*/
})
.catch(err => {
console.log(‘something went wrong’);
});
}

Now we are almost done. Let’s deploy the cloud function which holds all the business logic of the scheduled meetings .Run command
firebase deploy .

Congratulations, you have successfully deployed your cloud function which will help you to create a chat room on exact start meeting time.

About Author:

Qss author image Shiv pratap is a full stack engineer currently working with QSS Technosoft. He has worked in Node.JS, GraphQL, Express.js, ReactJS, Java, Twilio, Google and Firebase cloud compute solutions. He is always ready to explore new and upcoming technologies.

About QSS:

QSS has a proven track executing web and mobile applications for its esteemed customers. The company has a core competency in developing and delivering enterprise level applications using Google cloud functions and cron jobs. To Know More...