We proactively manage your software environment around the clock. The more visibility and monitoring there is in place, the more comfortable and easy it is to grow, innovate, save costs and assure quality as we described in this blog. Our answer to managing the software effectively and efficiently is intelligence and automation. 

Operations Management

Let your cloud infrastructure and applications be managed by our specialists to let your teams work in a safe and efficient environment that is being guarded day and night. Maintenance includes monitoring of processes such as:

Data pipelines (Ops management)

A data pipeline is an infrastructure component that ensures the delivery and necessary transformation of data. The basic health indicators are:

• Status of the job: success or failure?
• Latency: how long does it take for the job to complete?

Data quality/quality assurance (ops management)

Raw data is used to form the basis for algorithms and machine learning processes. Wrongly matched or enriched data are unreliable, and when used in business processes, they impact the ability to make informed decisions about your business. The quality of the data needs to be in an implacable state to be used in critical decision processes. During maintenance, data is checked for:

• Freshness  - was it delivered on time?
• Data completeness - do we receive all the files?
• Data accuracy - do the files contain the information we expected and in the appropriate format?

Cost management

Maximize the potential of the cloud while you keep in control of the cost. Monitor any significant changes in expenses that may appear when running dataflows. In case of any suspicious spikes in the billing report, we would investigate:

• which environment incurred the expenses (dev, test, or prod)
• which Google Cloud products/resources cause it
• which processes from these products are involved
  
  

Identity and access management

Adopting ‘the principle of least privilege’ we actively manage the access management of users to sensitive information. Our monitoring solutions let you easily manage individual users. 

Fine-grained access control and visibility for centrally managing cloud resources. IAM authorizes who can take action on specific resources, giving you full control and visibility to manage Google Cloud resources centrally. For enterprises with complex organizational structures, hundreds of workgroups, and many projects, Cloud IAM provides a unified view into security policy across your entire organization, with built-in auditing to ease compliance processes.

The maintenance process

Reports and automated alerts

Get notified to check data accuracy, when the tables were last updated when Dataflow is lagging behind or has failed via an automated email. Use visualization applications of your choice for maintenance and monitoring, such as Looker, Tableau, or Google Data Studio. The last one allows you to share reports with other team members for forecasting resource usage and costs and it’s free of charge.

Daily monitoring

The daily monitoring tasks include checking the following:

• Dataflows
• Cron jobs
• Scheduled queries
• The status of open Jira tickets
• File delivery in certain buckets

Monthly Service Review

A best practice is to take the maintenance to a more strategic level by reviewing periodically with senior IT management. As a result of this review, structural improvements can be made. These are the elements we typically  cover in a service review:

• Action points/minutes from the previous review
• Maintenance activities and tools
• Incident management
• Service requests
• SLA (service level agreement) overview
• Maintenance meeting - Topics & Minutes
• Billing report
• Service Improvement Plan