Brief Introduction and Clarification of Terms

In my view, Platform Engineering with DevOps involves the platform engineering team supplying and managing the essential stack for developers. This allows developers to access a service repository as a service. A DevOps culture is fostered through shared tools like GitOps with Argo CD, enabling the platform team to deploy the stack while the development team deploys their services.

A self-hosted Internal Developer Platform can facilitate the creation of a portal, utilizing solutions like Backstage that allow for the establishment of self-hosted portals through a framework. As a platform engineer, my responsibility lies in operating the portal and integrating the existing infrastructure.

In contrast, a Software as a Service (SaaS) Internal Developer Platform can also offer a portal that consolidates your services and infrastructure. Here, I refer primarily to SaaS solutions, where the platform engineer does not manage the portal but focuses on integrating the current setup to empower developers.

What Do We Mean or Understand by a Portal?

While it may appear promising (Fig. 1), the reality is somewhat different.

From a platform standpoint, we do not consider the portal to be an Internal Developer Platform. Instead, it serves as an interface that overlays an IDP, providing the necessary tools to leverage the IDP effectively. Essentially, the portal functions as the front end of the IDP.

We seek an API-based IDP that operates as a control plane, automating tasks that the platform team currently manages manually. This implies that customization occurs outside the IDP, which merely provides the API to facilitate these customizations within the portal.

Let’s break this down.

Imagine a typical configuration in various companies:

The platform team establishes a template that encompasses multiple steps or technologies to deliver a stack, including a Kubernetes cluster, a managed database, a Vault, and an ingress controller. Acting as the control plane, the platform team determines which template to implement for a project, either manually or via CI/CD. Meanwhile, the development team utilizes the provided stack as a black box, concentrating on deploying their services.

Now, let’s examine the scenario when the portal assumes the responsibilities of the platform team:

Rather than submitting a ticket, developers can use the Portal to provision the required stack through a self-service model and subsequently deploy their services. But how are templates and specific company requirements integrated into the portal?

This is where the platform team comes into play, enhancing the IDP to incorporate relevant templates within the Portal.

This should clarify things, but there will likely be questions regarding the nature of a portal and its distinction from or relationship to the IDP. I will attempt to clarify this in the next section.

The Relationship Between the Portal and IDP

We view the portal not as an Internal Developer Platform but rather as an interface that builds upon an IDP, providing all necessary components to integrate the IDP into the portal.

To reiterate: we are in search of an API-driven IDP that functions as a control plane, managing tasks currently handled manually by the platform team. This means that customization happens outside of the IDP, which simply offers the API for integrating these customizations into the portal.

To elucidate this further, let’s examine this diagram along with the associated requirements for a portal:

The portal should meet the following requirements:

• Self-Service: Enabling users to select services through templates without needing to submit tickets.
• Kubernetes Overview: A summary of all deployed clusters, providing meta-information about which services are operational and which are not.
• Standard Catalog: A repository of essential services, such as Kubernetes, S3 Buckets, databases, etc.
• Software Catalog: Allowing developers to integrate their services into the portal via the IDP.
• Documentation: Integrated or linked resources for services provided, ensuring users know how to utilize them.
• User Interface (UI): Streamlining access to essential functions, such as service overviews, costs, or custom dashboards.
• Template/Blueprint Functionality: For combining various technologies and deploying them using templates.
• SSO/OIDC: Allowing users to utilize their identity provider, with user management handled by the chosen provider.
• Organization/Tenant Capabilities: Ensuring users do not interfere with each other's projects.
• API and Interface Integration: Facilitating integration with the IDP, supporting a declarative Infrastructure as Code (IaC) approach.
• Logging and Monitoring: Providing insights at the meta level for the portal, with service-specific alerts managed elsewhere, such as via Kubernetes with Grafana.
• CI/CD Pipelines: Integration with CI/CD tools like GitHub for automated pipeline setups compatible with existing repositories and build processes.
• Cost Overview: Tracking expenses, such as the cost associated with a template or service.
• Security and Compliance: Ensuring the portal adheres to company policies and meets compliance requirements.

Moreover, the portal should be offered as either a SaaS or self-hosted solution, with engineering efforts concentrated on enhancing the IDP rather than managing the portal itself. It should act as a control plane that consolidates the most critical functions and interfaces, facilitating the integration of the underlying IDP.

Now that we have a clear understanding of what the portal should offer, we can assess how the IDP contributes to bringing the portal to life. The critical concept here is Plugin integration.

To grasp this better, let’s refer to this diagram:

The portal integrates with the IDP by allowing various technologies to be utilized, including template definitions. This is why Backstage is recognized as an open-source framework for building developer portals.

Logic and extensions, such as Terraform modules or Helm charts, should be developed externally to the IDP, with the IDP incorporating them as plugins. This maintains flexibility without necessitating a domain-specific language.

Plugin integration allows APIs from cloud providers to be leveraged for template creation.

The following diagram illustrates the interaction with managed services via API consumption.

Thus, we assert that we are looking for an API-driven IDP that functions as a control plane, assuming responsibilities currently managed by the platform team and making them accessible through a portal. The APIs and plugin integration are frequently grouped together and categorized as platforms. We differentiate them to separate framework tooling from the actual APIs utilized for service consumption.

These portals are heavily developer-focused. It is common to hear platform engineers discussing blueprints/templates as enablers for developers. However, we believe that portals should cater not only to developers but also to other stakeholders. In the next section, we will explore how different stakeholders might interact with the portal and their respective requirements.

Stakeholder Interaction with the Portal and Their Requirements

Various stakeholders can utilize the portal. We have narrowed down the stakeholders to focus on business service perspectives, primary users, and operators, excluding roles like Security, FinOps, Compliance, and Governance Officers, who nonetheless play crucial roles in gathering diverse requirements.

We will concentrate on four distinct stakeholders, starting with a global Product/Service Owner.

Global Product/Service Owner: This role seeks a comprehensive view of the portal as a service, including associated costs and potential capacity constraints. Requirements may include:

1. Monitoring platform status (Running, Failure, etc.) to stay informed about availability and overall health.
2. Identifying the disparity between utilized and available capacities across applications and the platform to manage capacity effectively and avert potential bottlenecks.
3. Informing application Product Owners about anticipated costs and any unusual alterations in projections, allowing them to respond proactively to cost changes.

Product/Service Owner: This individual is focused on the services they manage, including associated costs and potential capacity limitations. Requirements may include:

1. Monitoring the status (Running, Failure, etc.) of their application to ensure its availability and health.
2. Ensuring that confidential information about their applications remains secure and private.
3. Accessing application monitoring directly from the IDP to avoid maintaining a separate collection of links.

Developer: This stakeholder is interested in utilizing services through a self-service model and offering their services via the portal. Their requirements may include:

1. Creating a standard application setup to efficiently build a consistent application infrastructure.
2. Expanding the IDP's standard service catalog with their custom services for seamless integration.
3. Ensuring CI/CD pipelines are part of the standard application setup for efficient automation of development and deployment.
4. Consuming all IDP services through self-service to enhance efficiency without waiting for support.

Platform Engineer: This role aims to provide a self-service approach to eliminate bottlenecks in projects. Their requirements may include:

1. Defining the specific tools, technologies, and standards included in a standard application setup, ensuring consistency with organizational requirements.
2. Utilizing various public cloud providers for application setups, offering flexibility and the best available services.
3. Integrating the existing Kubernetes service catalog into the IDP for easy access to Kubernetes services through a unified interface.
4. Enabling users to fully utilize the IDP in a self-service manner to avoid becoming a support bottleneck.

Other stakeholders and their potential requirements may include:

Chief Information Security Officer (CISO): This individual focuses on maintaining security and compliance across all systems. Their requirements revolve around visibility, control, and adherence to organizational policies. Possible needs include:

1. Gaining transparency over the services being utilized to monitor usage and ensure compliance.
2. Automatically analyzing known vulnerabilities in utilized services and libraries to proactively address security concerns.
3. Enforcing governance rules regarding network and infrastructure policies through the IDP to maintain compliance.
4. Setting up alerts for rule violations to reduce the need for constant monitoring of the IDP.

Below is a potential interaction diagram between stakeholders, the portal, and the IDP:

As illustrated, various stakeholders can benefit from engaging with the portal.

Numerous IDP solutions are currently available, offered either as self-hosted, SaaS, or both. The following list presents some options:

• Kubermatic Developer Platform based on kcp
• Giant Swarm's IDP based on Backstage
• getport.io (port)
• Qovery
• OpsLevel
• Humanitec
• Mogenius
• Backstage
• Any other solutions that meet the requirements?

After considering diverse perspectives and experiences, what is my personal stance on this subject? Let’s explore.

My Personal Opinion on the Topic:

I believe that this topic may be overhyped in many organizations, and many companies are unclear about their needs. I am not referring to companies like Spotify, Apple, or smaller firms that are deeply embedded in Cloud Native practices, but rather to those trying to adopt it, chasing trends without a core business focused on IT services.

I could be mistaken, and I welcome your corrections! After several discussions over the months, I currently hold the following viewpoints:

1. A portal paired with an IDP does not have to be exclusively designed for developers; it can also serve internal platform teams by enabling them to provide a self-service stack.
2. Implementing a portal alongside an IDP necessitates a cultural shift; roles must be clearly defined, and requisite skills must be present within teams. You may think, “But I only click a template, and the responsibility for its functionality lies with the template provider/builder.” While this is true, developers can still misconfigure the offered service, leading to issues, such as defining an incorrect PodDisruptionBudget (PDB). If upgrading to a new template with a different Kubernetes version fails due to these PDBs, who is liable? The new template version may work seamlessly, but upgrades can become problematic.
3. If your teams are small, introducing extra complexity with an additional layer may not be necessary, especially if only a few projects are added annually. The effort required to construct an IDP may outweigh the benefits of self-service through a portal.
4. For organizations with over 500 developers and a small platform team, a portal combined with an IDP is sensible; otherwise, the platform team will struggle to keep pace with project demands and support requests. Scalability becomes a key issue.
5. Even when opting for a SaaS portal, a team will still be required to incorporate your company-specific stack and requirements into the IDP. An all-encompassing solution seems unlikely; however, a standard catalog that eases entry is feasible. Providers can assist in integrating your services into the portal, as reflected in the vision behind port (see the interview with CEO Zohar Einy), emphasizing the provision of tools rather than dictating portal design.
6. Many development teams are not interested in accessing a catalog or provisioning services through self-service. Even the one-click solution may seem excessive; they prefer to concentrate on development and related contexts, managing something akin to repos as a service. After development and deployment, the remaining processes should be managed through GitOps.
7. Constructing multi-tenancy necessitates special attention because multi-tenancy within the portal does not imply multi-tenancy at the cloud provider level. In simple terms, while users may not see other teams' projects in the portal, everything is deployed within one subscription at the endpoint (e.g., Azure), granting everyone contributor rights on that subscription. Thus, it is vital to ensure that the portal you select has mechanisms to detect this through policies in advance.
8. The integration of proprietary services via a plugin mechanism should be straightforward, without requiring the learning of a new domain-specific language or the need to write TypeScript code to integrate services into the Portal.

Wrapping Up

Determining whether to implement a portal based on an IDP within a company is complex. As discussed throughout this article, various factors, including team size and skill sets, come into play. The fundamental question remains whether this is merely a fleeting trend or if it will evolve into an industry standard justifying investment in innovation—despite current minimal benefits or potential additional efforts.

Honestly, I cannot provide a definitive answer to this question. As a platform engineer, I find the concept of a portal worthwhile, particularly if it does not necessitate self-hosting. This is due to the specific knowledge and resources required for portal integration and construction.

Unless you possess the necessary resources to establish and maintain it long-term, I would advocate for a portal solution like getport.io, DKP, etc., offered as a SaaS option. This approach allows for the integration of your services while enabling your resources to focus on onboarding stakeholders without the burden of managing the portal.

What is your perspective on this topic? Is it a mere trend, or does it offer genuine business value?

Contact Information

If you have questions, wish to engage in conversation, or simply want to stay connected, skip the comments on Medium and connect with me on LinkedIn. Don’t forget to subscribe to the Medium Newsletter so you never miss an update!