Using #DataOps Kitchens to Accelerate Analytic Cycle Time

Learn how Kitchens within the DataKitchen's DataOps Platform help data and analytics teams accelerate their analytic cycle time.

----------------------------------------------
[TRANSCRIPT]
Today, enterprise data and analytic teams face many challenges that hinder agility and innovation.

Most data teams need weeks or even months to get access to data sets, spin up environments, and deploy changes.

Central IT teams struggle to balance agility with control for self-service users.

And finally, most data teams have difficulty collaborating due to complicated organizational structures. Different teams use different tools, and work in different locations.

The DataKitchen DataOps Platform improves analytic agility, increases innovation, and fosters collaboration.

The Platform automates the creation of analytic environments so that developers and other self-service users can quickly, confidently, and safely experiment and develop new data products.

Here’s how it works. In the platform, users perform all work within virtual workspaces called Kitchens.

Kitchens are where teams build, manage, and run pipelines that provision infrastructure and do data work. Kitchens are like factories that contain assembly lines called Recipes.

Kitchens are hierarchical in nature, and users can easily create child kitchens, generating exact copies of the parent kitchen's recipes.

Here, the user creates a child kitchen by naming it, selecting a few settings, and running ingredients which automate the provisioning of infrastructure. In this instance, the user selects an ingredient that will create a new schema named after the kitchen, as well as add tables and load test data.

When finished with selections, the user creates the child kitchen. The new child sandbox kitchen looks exactly like the parent kitchen but now the user can work safely without impacting other developers or production workflows.

Once created, Kitchens contain everything users need to create and innovate, such as user access, tools and data, infrastructure connections, security, pipeline recipes, and commit history.

When a child Kitchen is created, the Platform also includes a git branch off of the parent Kitchen to enable version control. This allows users to merge their work safely without breaking anything.

When the technical environment underlying Kitchens match, it greatly eases the transition of analytics from development to production or between co-workers working in different locations or data centers.

In this case, the user adds a new data source to the graph and updates the recipe in the child kitchen.

After running and testing the pipeline, the user can merge the changes up to the parent kitchen.

During the merge process, the user reviews and resolves merge conflicts, similar to performing a Git merge.

When complete, the same recipe in the parent kitchen contains the new data source.

If the ideas in a Kitchen are no longer needed, the Kitchen can be easily deleted to keep everyone organized.

When deleting a kitchen, the DataKitchen Platform can also clean up the associated resources, such as the schema created earlier.

By using Kitchens, self-service users are confined within the framework of good development practices. The data team never loses control of the code and configuration files. If an idea doesn't work out, access can be easily revoked or the Kitchen deleted, while good ideas can be promoted into production.

When everyone is working in Kitchens, the ability to collaborate is significantly improved.

Individual users or teams can work independently in different locations, accessing different data centers or cloud resources, and using their preferred tools, but then collaborate and merge their collective work with confidence.

A single system-level view of the entire analytic process also means that everyone can see how their work impacts the whole.

Kitchens are a unique and exciting feature of the DataKitchen DataOps Platform.

Teams that follow DataOps principles and organize their work in Kitchens experience dramatic improvements in agility and innovation - creating a significant competitive advantage that sets them apart from their peers.