DataOps - Next only to DevOps

When the dust is going to be settled with DevOps-Development and Operations works hand in hand, another brand new technique to cache in on already on the go. Yes the DataOps-Data managers and Operations to work hand in hand. The advent of internet of things and machine learning in Industry 5.0 need decisions from both man and machines. This means the data experts constitute the current engineers turned data scientists. Data experts are no one than the experienced data engineering who are domain expertise in the former dates as subject mater experts.

Coming to the Operations part, the application of Agile to applications derived DevOps. So do the application of Agile to data will be the basis for DataOps. Furthering the dashboards been operated by silos there is a need for central operators who take the ownership of pulling up the data across without misinterpretation when the data crosses department boundary. Meaning the master data days are gone when one system operates one master and hence the dashboard were siloed which worked well for automating the pipelines of DevOps. When the data based decisions are the primary rule for any two decades we need this unified operating mechanism of giving value to the experts in the domain who can orchestra the data not only for their use but for the cause of bigger good.

Catalyst for DataOps is turning the data debts-running loose portfolios to diversify in markets resulted in silo data collection and interpretations., rather than as data assets-managing data proactively, carefully from day one. Example could be the data rich Facebook, Google, Netflix, and Amazon. Here there is no question of integration as the data is treated more carefully and secure. This is in contradiction to the decade old practice of creating enterprise data warehouses that dealt with nasty ETL processes. It would have worked for the automation revolution but need tweaks for the industry revolution.

Excerpts from : Andy Palmer, Michael Stonebraker, Nik Bates-Haus, Liam Clearly, and M. M. (2019). Getting DataOps Right. In O’Reilly Media.