Alert Members in Analytics Team and Business Stakeholders of any Database Changes in your Snowflake, SQL Server and Oracle databases.

In a large organization, data analytics development teams consist of several members. Most often members are located across multiple regions, locations and time zones. In addition, as the line between business stakeholders and technical development teams gets blurred, many times it becomes essential for stakeholders to be informed of any change in database structure and schema as soon as any change is deployed to the database.

As the analytics team tries to automate the CI/CD process, schema compare and database change deployment, it is not always easy to keep everyone informed of any change and all the details of the schema changes. Without an automated process, the team tries to inform via team meetings or conference calls or an email which is not always possible, often delayed and also not detailed enough.

If all the members in the team are not informed of any changes to the database structure then it creates confusion, duplicate work, and sometimes errors in development.

Before we look at how to send automatics notifications for any CI/CD change, lets review how to automate CI/CD process, Schema compare and database change deployment. I have written this in my previous blog (XYZ….) but let me summarize the process again.

4DAlert — Cloud based solution that automates DataOps, CI/CD processes, schema compare and database change deployment

Your Current Half automated and Manual Process — Not So good Process

Most analytics teams adopt an internal CI/CD process which in many ways is nowhere close to automated !!!). Members extract their DDL scripts manually from database, add all the script in sequence into text file, a senior member(whom we could call as change manager) takes all the scripts and consolidates the scripts into one single file and then upload the scripts into DevOps or Github folder and then pipeline take the script and executes the changes.

Drawbacks in the above process

1)Process is nowhere close to automation.

2)Generating scripts, putting all scripts in sequence, a designated change manager reviewing each script and consolidating all into one script are highly labor intensive and error prone.

3)As the process is not automated, so teams move the changes infrequently, causing delay in new database change deployment and realizing the value of the data and investment.

Automated CI/CD, Schema Compare and Change Deployment Process with 4DAlert

4DAlert, a cloud based solution automates CI/CD, Schema compare and database change deployment for Snowflake, SQL Server, Oracle and many other databases.

a)Solution provides an intuitive user interface for the development team to compare schema, identify the changes and object dependencies.

b)Developers could leverage the user interface to choose the changes they want to move to higher landscape and push the changes to source control tools such as Github or Azure DevOps.

c)Solution follows a DECLARATIVE method i.e. at anypoint it would move the most recent CREATE DDLS scripts to source control and at any point when ready to deploy the changes to target database, it compare the most recent CREATE script stored in source control with the target database and generate the ALTER scripts for deployment.

d)Once the deployment script is generated, the changes are deployed in the target database in automatic fashion.

e)Solution also comes with a set of APIs. If organizations want to integrate the process into any of their own devOps process, they could leverage the 4DAlert APIs to push the code to source control, generate deployment scripts and then deploy the changes using 4DAlerts’s secured APIs.

How could we automatically send notification of any schema changes after each deployment?

As organizations deploy the 4DAlert solution to automate the CI/CD, schema compare and automatic change deployment, team members and business stakeholders could subscribe to the notifications process. As soon as CI/CD process deploys the changes, the solution would automatically send the copy of all DDL scripts and deployment scripts and their status to show whether it passed or failed.

Above notification process keeps everyone involved informed, avoids duplicate effort and moreover gives clarity on what changes were deployed. This way the development team, business stakeholder and end users are immediately aware of the changes and start leveraging the new functionality immediately after that. This process democratizes the CI/CD process, keeps the process transparent and increases productivity.

Conclusion

In order to bring transparency, keep everyone informed of any deployment and democratize the database schema change deployment, it is essential for the organizations to automate the CI/CD process end to end and have an automated notification process. This process allows every member of the team to understand the DDL changes deployed so that they can adjust their work, avoid duplicate work and minimize the confusion of schema change process. This automation increases overall productivity of the team, deliver functionalities much faster, and helps organizations get most value out of the analytics investment.

To see a live demo, please contact experts are 4DAlert as support@4dalert.com or visit our website as https://www.4dAlert.com

Author — Nihar Rout, Managing Partner and Chief Technology Strategist At 4DAlert