Parallel Development at Scale: How to Achieve It

Many organizations do parallel development — especially if they’re developing large, complex products. Here, we cover the basics, pros and cons, and how to do parallel development at scale.

Read along or jump to the section that interests you the most.

Parallel Development Definition and Overview

Parallel development is working on multiple projects or features at the same time.

This includes:

• Multiple teams.
• Parallel releases.
• Multiple variants.

For example, if you’re building large, complex products, you probably have multiple teams working on multiple pieces in parallel. This includes multiple component teams and feature teams. It also includes front-end, back-end, API, and database teams — among others. You might also have teams working on parallel releases or multiple variants.

There are many advantages to parallel development. But it is also complex and can be challenging to manage.

Parallel Development Pros and Cons

There are some pros and cons you need to weigh when considering parallel development.

Organizing and orchestrating the work of all of these teams simultaneously is challenging. Their changes are interdependent. And all too often, these teams push integration and integration testing to the end.

But by waiting until the end, they cause delays in production, require massive rework, and create quality issues.

It’s a challenge to get parallel development right. But, when you do, you can avoid the cons and get the benefits of the pros.

So, how do you do it?

What Helps in Parallel Development?

Branching and testing are both critical to the success of development in parallel.

At the end of the day, all development patterns are managed with branching, merging and tagging.

When branching is manual, it hinders parallel development. That’s because branching is:

• Error prone.
• Hard to learn / remember (often on a whiteboard).
• Many companies write scripts to manage (which is costly).

When branching is automated, you can have different teams work independently on related codebases without overwriting each other’s changes and yet have an automated process for integrating their changes, early and often. This enables teams to work simultaneously, without causing delays in production. They’ll be able to more easily integrate their changes and build on each other’s work, without massive rework.

Testing continuously throughout the development cycle ensures that you can quickly identify the root causes of failure. This helps you avoid quality issues while accelerating development.

This is especially important if you’re trying to do development at scale for large, complex products.

Throughout this blog, we’ll focus primarily on using branching for parallel development.

How to Do Parallel Development at Scale With Version Control

To do parallel development at scale, you need the right version control solution complete with the right branching capabilities.

That solution is Helix Core, version control from Perforce. Helix Core offers Perforce Streams for branching. By using Helix Core and Perforce Streams, you can do parallel development at scale.

Here’s how.

1. Get Helix Core

Helix Core is the best version control for world-class development at scale. It can handle all of the dimensions of scale:

• Massive parallel development.
• 10s of 1,000s of users.
• Unlimited numbers of files.
• Petabytes of data.
• Geographically distributed development across the globe.
• Massive monoliths.
• Complicated projects (e.g., component based development).
• Large numbers of variants (e.g., space and time).

That makes Helix Core an ideal candidate for teams working on large, complex projects with parallel development.

You can test Helix Core out for free for up to 5 users and 20 workspaces to see how it works.

Try Helix Core

2. Use Perforce Streams

Perforce Streams is branching in Helix Core. Streams provides automation, flexibility, and control for development in parallel.

Streams:

• Makes it easy to see (visually) how teams relate to one another.
• Encourages integration.
• Automates the separation and then integration of multiple teams.
• Shows you when changes are ready to integrate from one team to another.

▶️ See for yourself how Perforce Streams makes this possible:

3. Design and Automate Development Processes

With Perforce Streams, you can design and automate your development process.

You’ll be able to:

• Define the branches and relationships graphically without writing scripts.
• Automate the workflow, so you don’t have to remember how it works.
• Reduce errors, because the process will be automatically enforced.

This supports your complex parallel development process and reduces bottlenecks.

4. Shift Left

The later a defect is found, the more difficult and expensive it is to fix. This causes massive rework — and massive delays.

By shifting left — using continuous integration builds and testing — you can find and fix defects earlier.

Here’s an example of how to do this using Helix Core with Jenkins.

There are multiple development teams working in parallel. That code goes gets tested pre-integration. Then it goes into the CI build, is integrated, and tested again.

And this is what your workflow looks like in Perforce Streams, moving from production to UAT and so on.

Mixing immature code with mature code hinders your process. But if you use Perforce Streams to keep immature code separate from mature code, you’ll streamline your process and keep it running.

You won’t risk a bad check-in breaking a build. And you’ll avoid the inevitable email asking everyone to stop checking in code to send the build to QA. Instead, you’ll be testing earlier and more often.

By shifting left, your developers will get faster feedback — and you’ll get increased velocity and productivity.

5. Integrate Early, Integrate Often

Within a team, it’s a best practice to merge early and merge often. But across multiple teams, it’s a best practice to integrate early and integrate often.

With Helix Core, you can integrate early and integrate often.

You’ll be able to:

• Organize and orchestrate work across multiple teams automatically.
• Make it easy to integrate earlier in the process and avoid rework.
• Accelerate development by addressing quality issues earlier.

This helps you save time by avoiding massive rework.

Here’s an example of how this works when using Helix Core with Jenkins.

There are three teams working simultaneously on a project. They’re able to integrate their changes with each other early and often throughout the development lifecycle. This helps them avoid a long release tail (which is the length of time between a code freeze and deployment).

These teams are also able to test pre- and post-integration. So, they can quickly identify the root causes of failure.

Plus, Helix Core gives them a single unified platform for their development projects. Developers working on each piece get faster feedback and increase their productivity.

Here’s what this workflow looks like in Perforce Streams:

If you’re managing multiple parallel releases, there’s some serious ROI in the time saved per defect and number of defects per year by using Helix Core and Perforce Streams.

Contact us to learn more about how much you can save with Streams.

Contact Us

7. Manage Multiple Variants

Your company might produce slightly different versions of a product. Most of the code is the same. But each variant differs.

So if you fix a bug or create a new feature for the common code, you’ll need to propagate it across multiple variants. For example, in the semiconductor world, you might have 100s of variants that need to be maintained and tested as base code changes.

But there are the same issues in managing multiple variants as in parallel releases. It’s a challenge to determine where the change needs to go. And it’s a manual process to perform and verify the change.

If you use Helix Core and Perforce Streams, you can manage multiple variants easily.

Let’s say you’re building a flash drive. You have multiple variants for different storage capacities — 32GB, 64GB, 128GB, and 256GB. The parent stream contains all the common code for the flash drives.

Your Stream inherits base/common code from “flash” stream. This contains all the specific code needed for the 128GB version. This makes it easy for you to propagate the change to the right Stream.

By using Perforce Streams in Helix Core, you’ll be able to reduce costs by:

• Easily determining where the change needs to go.
• Automatically performing and verifying the change.

Do Parallel Development at Scale With Helix Core

Helix Core can handle all dimensions of scale in development. Multiple teams. Parallel releases. Multiple variants. And so much more.

That’s why Helix Core is leveraged by some of the world’s biggest development teams who build some of the largest, most complex products. This includes 9 out of the top 10 semiconductor companies.

Using Helix Core helps you do parallel development at scale. And Perforce Streams makes it possible.

With Perforce Streams, you can:

• Automate any development process.
• Leverage flexible workflows.
• Maintain control.
• Simplify complex environments.

And you’ll gain:

• Increased velocity/productivity.
• Improved quality.
• Lowered risk.
• Increased ROI.

See for yourself how you can leverage Helix Core and Perforce Streams. Get started for free for up to 5 users and 20 workspaces — and scale it up to handle your development process.

HELIX CORE FOR PARALLEL DEVELOPMENT

Related Content:

• Learn about using tools to find code vulnerabilities, ensure standards compliance, and reduce time-to-market early in the development process with Perforce's Shift Left 101
• Version Control Branching
• CI CD Process
• How to Optimize Your Software Delivery Pipeline
• The Essential Guide to Parallel Testing