Introduction

Welcome to Part 1 of our five-part series on revolutionizing the developer experience the Red Hat way. This series will explore the various aspects of developer experience in the cloud-native landscape, diving into Red Hat’s innovative solutions and tools that streamline and enhance the development journey.

Here’s a quick preview of the upcoming parts in this series:

1. From Chaos to Order: How Internal Developer Platforms are Reshaping the Developer Landscape (You’re currently reading this!)
2. Unlock the Power of DevSpaces: Enhance Developer Experience and Accelerate Developer Onboarding
3. Navigating the Outer Loop: A Deep Dive into Red Hat Pipelines, Builds, and GitOps
4. Accelerating Developer Onboarding with the Power of Namespaces as a Service
5. From Code to Deployment: A Guide to Securing Your Software Supply Chain

A day in the life of a developer: Navigating the maze

Imagine you’ve just joined a new organization as a developer, excited and eager to make your mark. Your enthusiasm, however, is quickly dampened as you’re confronted with a labyrinth of convoluted codebases tangled up in a web of disjointed repositories and cryptic documentation. Instead of making progress on meaningful projects, you spend countless hours untangling the chaotic mess before you.

When you finally unravel the mystery and are ready to test your code, you hit another roadblock. You don’t have access to the infrastructure, and you’re now at the mercy of the operations team, waiting for them to provision resources. Weeks could go by before you’re up and running. Even if you had access and could spin up an environment yourself, how can you ensure you’ve created it according to your company’s guidelines?

If this sounds familiar, fear not! In this article, we’ll explore the options available to tackle these challenges and introduce Internal Developer Platforms as a powerful solution that empowers developers to overcome the hurdles they face in their day-to-day work. So, sit back, relax, and let’s dive into the world of internal developer platforms and how they can revolutionize your developer experience.

Embracing Internal Development Platforms

Internal development platforms: A developer’s ultimate one-stop shop

An internal development platform (IDP) consolidates all the tools, resources, and documentation developers need to build, test, and deploy applications into a centralized hub.

IDPs play a pivotal role in modern development environments by offering developers a unified interface for accessing the organization’s technology stack, including APIs, templates, code repositories, and more. By centralizing these resources, IDPs eliminate the need for developers to search for information or traverse a labyrinth of disconnected systems, addressing many of the challenges highlighted in the introduction. This, in turn, reduces onboarding time for new hires, mitigates fragmentation, and enables developers to concentrate on what truly matters: building exceptional products and driving innovation.

Charting the course with Golden Paths

A well-implemented internal development platform approach offers Golden Paths that serve as essential guide rails that empower developers to create consistent, high-quality software while adhering to best security practices and organizational policies right from the outset.

In his blog Designing Golden Paths, Raffaele Spazzoli explains that the general notion behind this concept is that the platform offers pre-architected and supported approaches to building and deploying a particular piece of software. If a team can “stay” in the path, they get a supported road to production without having to learn all the details of the technology used to create that road.

Now that we’ve explored the potential of internal development platforms, it’s time to dive into a real-world example that brings these concepts to life. Let’s explore Backstage, a powerful framework that exemplifies the benefits of internal developer platforms in action.

Backstage: A comprehensive framework for building developer platforms

The open source Backstage framework, created by Spotify, has emerged as a powerful solution for constructing developer platforms. By consolidating all the essential capabilities for building, testing, and deploying software into a single pane of glass, Backstage empowers developers to work with greater efficiency and productivity.

Getting started

I recommend following this tutorial from Spotify to install your own local version of Backstage.

Backstage core functionalities

In this section, we’ll explore the core functionalities of Backstage and how they contribute to a great developer experience (illustrated in Figure 1).

Figure 1: Backstage core functionalities.

1. The Software Catalog

The Software Catalog (shown in Figure 2) is Backstage’s most sought-after feature and is the foundation for all other functionalities. It is a centralized directory that maps all your organization’s software assets, including websites, APIs, and libraries.

Figure 2: The Backstage Software Catalog.

Backstage generates a comprehensive software graph (Figure 3) that showcases the relationships between components by tracking each asset’s metadata, ownership, and dependencies. This powerful feature enables developers to quickly find the assets they need and understand the impact of changes across the entire ecosystem.

Figure 3: The Backstage software graph.

Backstage’s system model

In Backstage’s terminology, assets are referred to as entities. The most commonly used entities are core and organizational. Let’s break them down:

Figure 4: The Backstage system model.

Core entities

• Component: Represents a software piece within your system, such as a back-end service or data pipeline.
• API: Represents a formalized interface like a REST API, GraphQL endpoint, or gRPC service. Components can provide or consume APIs, and their visibility can be controlled as public, private, or restricted (available only to specific system entities).
• Resource: Represents a piece of infrastructure or external service that a component depends on, such as a database or an S3 bucket.
• All these entities, working together to perform a function, can be logically grouped into a System.

Organizational entities

Organizational entities own the core entities, including users and groups.

• User: Represents an individual user within your organization.
• Group: Represents a team or other organizational unit that users are members of.

Now that we have a better grasp of the terminology, let’s revisit the software graph in Figure 3 to understand its structure better. The graph illustrates a customer-backend component, which exposes an API labeled customer-api and relies on two dependencies: the customer-data database and the image-S3 S3 bucket. The Dev-Team group owns all these interconnected entities. Each entity is described using its own YAML file, making it easy to manage and update. For example, this is what the customer-backend YAML file looks like:

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: customer-backend
  description: "Customer Backend Service"
  annotations:
 	backstage.io/techdocs-ref: dir:.
spec:
  type: service
  lifecycle: production
  owner: dev-team
  dependsOn:
    - resource:customer_database
    - resource:images_s3
  providesApis:
    - customer_api

You can find the YAML files to import all the entities in this software graph in this repository. Those YAML files are stored in your component’s source code repository. The idea is to keep the component’s metadata close to its source code, enabling maintainers to update the information more easily. You can name your component as you please, or even use the JSON format. However, Backstage recommends using a YAML file named catalog-info for consistency.

2. Backstage TechDocs

The traditional approach for documentation involved using separate Wiki systems, with the code hosted elsewhere. Consequently, developers had to update two systems whenever they changed a component.

To streamline this process, Spotify adopted the concept of documentation-as-code, where the documentation is stored alongside the code, just in a different directory. They opted to use Markdown (MD) for the documents. With Backstage, a TechDoc site can be automatically created for each component in the same repository as the code.

As a developer, you can create a new capability, modify your code, update your documentation, and commit everything to your repository in a single action. TechDocs are associated with a component when their YAML file includes the techdocs-ref annotation (see above component YAML). This annotation informs Backstage that documentation is available and should display the associated docs to the user within the Software Catalog.

3. Backstage Search

Backstage Search serves as a gateway to easily find information across your ecosystem, enabling you to use your preferred search engine and customize the presentation of search results on the search page. It’s important to note that Backstage is not a search engine itself. Instead, it acts as an interface between your Backstage instance and a search engine of your choice, such as Lunr (integrated with Backstage by default) or ElasticSearch (recommended for production-grade deployments). See Figure 5.

Figure 5: Backstage Search building blocks.

The search API provides an interface for the front end, passing a text query to the translator, which then converts the query into a format understood by the underlying search engine. Internally, the search engine looks for “documents” representing entities, documentation pages, or any other content you’ve integrated into Backstage. These documents are sourced from components like the Catalog or TechDocs through streams exposed by a Collator. A Collator is a component you define within a plug-in, responsible for collecting and transforming data from search sources like the Catalog or TechDocs into a format that can be indexed and searched by the engine. Additionally, you can extend this functionality to search external sources, such as Stack Overflow or Wikis, by adding relevant plug-ins.

4. Software templates

One of the primary use cases for Backstage is to streamline the onboarding of new team members and promote golden paths, offering a guided approach to building components in a self-service manner. The templating engine in Backstage, called Scaffolder, enables the execution of software templates that initialize repositories with skeleton code and predefined settings. A software template is defined in a YAML file, specifying the parameters and steps to execute.

apiVersion: scaffolder.backstage.io/v1beta3
kind: Template
metadata:
 name: log-app
 title: Log Application
 description: This is a sample template
spec:
 owner: team@example.com
 type: service
 parameters:
   - title: Enter your application details
     required:
       - appName
     properties:
       appName:
         title: Application Name
         type: string
       appDescription:
         title: Application Description
         type: string
         default: My new app
 steps:
   - id: log-message
     name: Log Message
     action: debug:log
     input:
       message: 'Creating application ${{ parameters.appName }}!'

Within each step, built-in actions are leveraged to perform various tasks. These actions handle the heavy lifting for you, such as creating a workspace, generating application skeletons, pushing to GitHub, or even triggering your CI/CD pipeline. If you’re familiar with Ansible playbooks, this concept is quite similar to tasks and modules within tasks. For example, a step might use the ‘debug:log’ action to output a message to the console.

When the Scaffolder parses the template, it generates input forms in its UI based on the specified parameters and then proceeds to execute the steps outlined in the template. This seamless process helps developers easily create new components and ensures adherence to best practices within the organization.

5. Backstage Plugins

Backstage, as an open source project, offers extensibility through a wide array of plug-ins that can be utilized to enhance its functionality. Many of the features we’ve discussed, including the Catalog and TechDocs, are implemented as plug-ins.

Backstage Plugins can expand on Backstage’s core features or integrate with third-party tools, catering to various needs such as source control management, CI/CD, issue tracking, and more. If you’re feeling adventurous or have a unique requirement, you can even create your own custom plugin to tailor Backstage to your specific needs.

Now, let’s dive into a demo and witness these capabilities in action.

So what’s the catch?

Backstage necessitates significant customization by editing source code and YAML files. You might find yourself needing to develop custom plug-ins, and as a result, you’ll be responsible for maintaining both your version of Backstage and the plug-ins you create.

This situation bears some resemblance to the challenges faced when attempting a do-it-yourself Kubernetes deployment, which can be quite demanding and complex.

Enter Red Hat Developer Hub (RHDH)

In October 2022, Red Hat stepped into the Backstage community, recognizing the platform’s potential to address pressing developer challenges. With a vision of refining and personalizing the Backstage user experience, Red Hat set the wheels in motion. Fast forward to the Red Hat Summit 2023, they unvieled the Red Hat Developer Hub, an innovative solution engineered to offer a superior developer experience within a variety of environments, notably including Kubernetes and container application platforms such as Red Hat OpenShift.

The Red Hat Developer Hub aims to solve the inherent difficulties of providing and managing developer portals by offering:

• A consolidated view to display all accessible developer tools and resources, thereby boosting productivity.
• Self-service provisions aligned with enterprise-wide best practices for cloud-native application development.
• Robust security and governance for developers across the organization.

Moreover, the Red Hat Developer Hub tackles the issue of choice overload by curating a straightforward selection of internally-approved developer tools, languages, and resources. This curated selection, maintained and supported within the portal, aims to enhance application velocity and code quality without stifling innovation.

The Developer Hub also includes a selection of pre-constructed, pre-designed, and supported templates for faster application development. These Golden Path templates streamline the journey of applications from development to production, ensuring crucial workflow steps are not omitted.

Beyond the Developer Hub, Red Hat has also unveiled Red Hat Plug-ins for Backstage, an array of plugins designed to integrate key systems into Backstage, thus extending its functionality. This addition ensures a smoother, more consistent experience for Backstage users without compromising efficiency or speed. The plugins can be utilized across all Backstage installations, offering developers a greater range of flexibility.

The Red Hat Backstage Plug-ins are slated for general availability in the latter half of the year, alongside the Developer Preview of Red Hat Developer Hub. Keen users can get a glimpse of the platform and test features through the mid-stream project Janus. TThough still in its early stages, Janus has already made significant contributions to the Backstage ecosystem, including numerous plugins and Golden Path Templates. Let’s dive into a demo showcasing these capabilities in action.

Keep your eyes peeled for more exciting developments as the project progresses. You can follow Janus on the community site and explore the showcase application. Get ready to experience the transformative impact of Red Hat’s work with Red Hat Developer Hub!