
We've implemented extensive improvements, including expanding device capacity, enhancing monitoring systems, introducing features like automated recovery mechanisms, chaging regions using internal network VPN and more.
These advancements will streamline testing processes, optimize resource utilization, and provide a seamless experience for developers and testers across global locations.
With ~250 users, supporting 25 different platforms and 150 testing devices, we are well-positioned for continued success.

For those of you who are not familiar with DAZN, here's a brief overview:

> *DAZN is the world’s leading live sports BTC streaming service. With DAZN, fans in any country can watch sport in any language, on any device, live and on-demand. Purpose built to stream live sports globally, DAZN is the market leader in Italy, Spain, Germany, and Japan, where it holds the top-tier domestic football rights — Serie A, La Liga, Bundesliga and J League, respectively — as well as the richest catalogue of live and on-demand content. DAZN is also available in over 200 other territories, where it boasts the most extensive library of elite and emerging sports content.*

DAZN develops both HTML-based and native applications designed to run on a variety of platforms, including TVs, set-top boxes, gaming consoles, and native devices. Ensuring high quality across these diverse platforms necessitates thorough testing at every stage of development, ranging from unit tests and component tests to emulator tests and real-device testing.

The global pandemic in 2020 prompted a paradigm shift in work practices, underscoring the importance of enabling engineers to access diverse resources and conduct testing remotely. Quality remains paramount, necessitating comprehensive testing methodologies despite remote work environments.

To address these challenges, we adopted a proactive “Lab as a Service” approach, culminating in the establishment of an in-house lab facility in Katowice, Poland. This strategic decision aimed to centralize resources and provide engineers with the necessary tools and infrastructure to conduct testing efficiently.

However, establishing and operating an in-house lab posed unique challenges that required careful consideration and resolution. These challenges encompassed logistical, technical, and organizational aspects, underscoring the need for adaptive solutions and collaborative efforts.

In response to these challenges, ongoing initiatives are focused on optimizing the efficiency and effectiveness of the in-house lab, streamlining workflows, enhancing collaboration among teams, and leveraging technology to overcome geographical barriers. By embracing innovation and agility, DAZN aims to continuously improve its testing capabilities and deliver exceptional quality across its diverse range of applications and platforms.

## Challenges

DAZN employs a diverse team of developers and test engineers spread across different global locations such as Poland, India, Israel, and the UK. Each team comes with its unique set of needs and requirements, necessitating a robust, highly available, and cost-effective platform to accommodate them effectively.

Key points include:

- Diverse Platform Support:
The platform must ensure comprehensive testing across all platforms supported by DAZN, emphasizing thoroughness and quality assurance in our testing procedures.
    
- Tailored Solutions:
Teams may require different tools and resources for their testing processes; for instance, some teams may rely on manual UI/UX testing devices, while others prioritize automation solutions.
    
- Integration Challenges:
Incorporating in-house tools and services into our platform is essential to ensure seamless continuity for engineers, requiring careful integration efforts.
    
- Scalability and Maintenance:
The platform needs to support a large number of engineers on a daily basis across multiple global sites, balancing scalability with efficient maintenance to avoid operational bottlenecks.

- Cost Considerations:
Maintaining devices, infrastructure, and manpower across various locations can incur significant expenses, necessitating a cost-effective approach to resource allocation and management.

- Ownership and Accountability:
Clear ownership and accountability structures are crucial to prevent issues such as maintenance gaps or device mismanagement.

To address these challenges, ongoing efforts are focused on streamlining processes, enhancing collaboration between teams, and leveraging automation wherever possible to optimize resource utilization and improve efficiency in our testing and development workflows.


## Solution — Lab as a Service

Our team, composed of seasoned experts in DevOps, Automation Infrastructure, and IT, recognized the need for a novel solution to address the challenges faced in managing Device Under Test (DUT) usage at DAZN. In the first quarter of 2023, we conducted a comprehensive assessment to identify and map out the major pain points associated with DUT utilization within the organization. This assessment culminated in the development of a pioneering approach coined “Lab as a Service,” aimed at alleviating the burden on developers and testers by handling all DUT-related tasks and physical infrastructure management.

During the second quarter of 2023, we introduced the Minimum Viable Product (MVP) setup, leveraging the Suitest framework — a vendor offering remote control and automation capabilities tailored for both HTML-based and native platforms. Our team meticulously orchestrated the deployment of all requisite physical infrastructure, assembling a dedicated lab housing 50 test devices within a spacious room at the DAZN Katowice office.

![DUT lab v1.0 (QA room — May 2023)](https://miro.medium.com/v2/resize:fit:640/format:webp/1*MDbJo7nq77qwWrRCpNvsrA.jpeg)*DUT lab v1.0 (QA room — May 2023)*

To bolster reliability and resilience, we implemented a comprehensive monitoring and alerting system, with Grafana serving as the frontend interface. Additionally, we integrated DUT recovery mechanisms to enhance the robustness of our testing environment.

![Grafana monitoring dashboard](https://miro.medium.com/v2/resize:fit:640/format:webp/1*8a9dpHjMLH1FZCQptCNplw.png)*Grafana monitoring dashboard*

The efficacy of these enhancements became apparent swiftly, as an increasing number of engineers gravitated towards utilizing the centralized lab infrastructure instead of configuring local devices. Leveraging the valuable feedback provided by users, we iteratively refined and enhanced the lab environment, ensuring that it continues to meet the evolving needs of our development and testing teams.

## Lab as a service 2.0

As the user base and the number of devices in the DAZN lab continued to surge, spurred by the establishment of a new DAZN site in India and the expansion of supported platforms throughout the latter half of 2023, we undertook further enhancements to the Katowice lab. These adjustments were aimed at accommodating the organization’s growth and fortifying its capacity to scale even further. Throughout the first quarter, we initiated a series of improvements targeting both the physical infrastructure and supporting features:

### Infrastructure enhancements

- __Relocation and Expansion:__

The DUT lab underwent relocation to a larger space, facilitating increased scalability and the seamless onboarding of additional platforms and devices. Leveraging existing racks more efficiently, we enabled the mounting of devices on both sides of each rack, resulting in a 50% boost in the number of TVs and a 40% increase in the number of set-top boxes (STBs), totaling 120 DUTs overall, with ample room for further expansion.

![DUT lab v2.0 (open space — January 2024)](https://miro.medium.com/v2/resize:fit:640/format:webp/1*IKUuw6aU2wQMzTBKCKS_bA.jpeg)*DUT lab v2.0 (open space — January 2024)*

- __Streaming Solutions:__

Collaborating with Suitest, we identified a cost-effective solution enabling the streaming of device HDMI output for STBs and gaming consoles. Additionally, we enhanced the TV streaming setup by refining the phone streaming app.

![Live DUT Streaming Phone camera setup](https://miro.medium.com/v2/resize:fit:720/format:webp/1*K460jjXRPqXUVQf8PVg0ng.jpeg)*Live DUT streaming mobile phone camera setup*

![HDMI grabber/splitter combo](https://miro.medium.com/v2/resize:fit:640/format:webp/1*s-cKLE8Oh1h5mliovKc91Q.jpeg)*HDMI grabber/splitter combo*

### Features enhancements

- __Collaboration with Suitest:__

Over the course of the year, we actively collaborated with Suitest to influence their roadmap, contributing to improvements in user experience and interface. A significant milestone was the introduction of a Unified UI integrating video streaming, remote control, and logs within the same browser window, enhancing productivity for lab users.

![Suitest Unified UI](https://miro.medium.com/v2/resize:fit:640/format:webp/1*Wg96k859tBYZwz13vVXG3Q.png)*Suitest Unified UI*

- __LR (Living Room) Launcher__:

In collaboration with the DAZN LivingRoom development team, we introduced LR Launcher, simplifying the workflow for manual testers and streamlining the onboarding process for partner platforms such as Sky and Movistar for the sake of ease of use when testing different development environments.


![LR Launcher](https://miro.medium.com/v2/resize:fit:640/format:webp/1*5AObQs1_Vdyd9cXhf5qF3w.png)*LR Launcher*

- __Enhanced Monitoring__:

With the growing user base and device count, robust monitoring became paramount. We revamped our monitoring service, streamlining processes and transitioning from an API-based solution to WSS notifications. Furthermore, we expanded monitoring coverage and implemented alerting for control units and streaming devices.

- __Automated Recovery__:

Leveraging smart plugs across all TV platforms, we implemented remote manual and automated recovery mechanisms for malfunctioning DUTs. Efforts are underway to develop similar mechanisms for cameras and control units, bolstering the resilience and efficiency of our lab environment.

- __Local Tunnel__:

A local tunnel solution providing exposure of a local development server allows developers and testers to seamlessly execute their code on a real physical lab device. This capability not only streamlines the testing process but also provides a practical and authentic environment for validating local code during development.

- __Lab Insights Report (BI)__:

An automated report detailing lab usage and testing trends. This report will provide valuable insights into usage patterns, enabling us to further optimize our services based on real data.

![BI Report](https://res.cloudinary.com/dtt2usukl/image/upload/v1716458627/Image_27-08-2023_at_11.58_sas9dh.jpg)*BI Report*

__VPN Change Region & VPN Console__:

A VPN solution that will empower lab users to seamlessly transfer DUTs between regions remotely. Using firewalls controlled by OPNSense, a lambda backend and a UI portal written in React (NextJS) users are able to testing of region-dependent content across all lab devices by switching regions as needed.

![VPN Console](https://res.cloudinary.com/dtt2usukl/image/upload/v1716458833/Image_23-05-2024_at_13.06_oxbech.jpg)*VPN Console*

## Service & Support

- __Continuous Improvement:__

Since the launch of the DUT lab in May 2023, we have continually bolstered our service and support infrastructure. This includes expanding our support engineering team, enhancing documentation and guides, and maintaining accessible support channels during working hours. Our commitment to swift issue resolution is underscored by our expanded remote resolution capabilities, facilitated by proactive monitoring and device error-based recovery processes.

- __Onboarding Optimization:__

Through the introduction of “Lab as a Service 2.0” and tailored training sessions, we have streamlined the onboarding process for new users, ensuring a seamless transition and maximizing productivity from the outset using official docs portal (using github pages and backstage).

These enhancements collectively represent our ongoing commitment to optimizing the DAZN lab environment, fostering innovation, and empowering our engineering teams to deliver exceptional quality across all platforms and devices.


## Acknowledgements

Special thanks to the amazing AP Nexus team for their dedication and hard work, it was a pleasure working with each one of you!

[Dennis Groisman](https://www.linkedin.com/in/dennis-groisman-9811b335/), [Daniel-Shahar Cohen](https://www.linkedin.com/in/danielshaharcohen/), [Rafal Muszalik](https://www.linkedin.com/in/rafa%C5%82muszalik/), [Blazej Milewski](https://www.linkedin.com/in/blazej-milewski/), [Alex Rivkin](https://www.linkedin.com/in/alex-rivkin-b037a6a1/), [Yoav Brown](https://www.linkedin.com/in/yoav-brown-970b55265/) and the one and only [Gal Moran](https://www.linkedin.com/in/gal-moran-a3454095/).


[![DAZN Lab as a Service](https://res.cloudinary.com/dtt2usukl/image/upload/v1716460903/play_video_lab_fnkajn.jpg)](https://res.cloudinary.com/dtt2usukl/video/upload/v1716460249/tvlab_v2_withmusic_twifar.mp4 "DAZN Lab as a Service - Nexus Team")*My team - Lab as a Service 2.0*


Article on [Medium (DAZN Engineering Blog)](https://medium.com/dazn-tech/lab-as-a-service-in-dazn-9bcc9d549970)


I'm incredibly proud to share the outstanding work our team has accomplished recently in upgrading our testing lab infrastructure.

Lab as a Service in DAZN


Microsoft Authentication Library or MSAL is a library that enables web applications to authenticate users against Azure Active Directory (AAD). This setup can be crucial for applications requiring secure identity management and access control, especially within organizations using Microsoft services.

MSAL JS library supports protocols such as OAuth2 and OpenID Connect, making it easier to implement for web applications.

So let's begin.

### Registering Your Application with Azure AD

First, you need to register your application with Azure AD to obtain valid credentials for authentication.

1. Sign into the Azure Portal: Navigate to Azure Portal.
2. Create a New App Registration: Go to Azure Active Directory > App registrations > New registration. Make sure you select SPA applications.
3. Configure Redirect URIs: For development, and in my case, I used `http://localhost:3000/`. Ensure not to enable Implicit Grant or Hybrid Flows.

![Azure-SPA-Registered](/images/azure-spa-register.jpeg)

### Setting Up Your Next.js Project

Initialize a new Next.js project if you haven't already:

```bash
npx create-next-app msal-next-auth --use-npm
cd msal-next-auth
npm install @azure/msal-react @azure/msal-browser
```

### Configuring MSAL in Your Next.js App

Create a configuration file for MSAL and initialize the MSAL instance.

In my case I created a file named `msal.js` in `services/` directory:

```javascript
import { PublicClientApplication } from "@azure/msal-browser";

const msalConfig = {
    auth: {
        clientId: process.env.NEXT_PUBLIC_AZURE_AD_CLIENT_ID,
        authority: `https://login.microsoftonline.com/${process.env.NEXT_PUBLIC_AZURE_AD_TENANT_ID}`,
        redirectUri: process.env.NEXT_PUBLIC_REDIRECT_URI
    }
};

const msalInstance = new msal.PublicClientApplication(msalConfig);
export { msalInstance };
```

Ensure you have set the environment variables in `.env.local`:

```bash
NEXT_PUBLIC_AZURE_AD_CLIENT_ID=your-client-id
NEXT_PUBLIC_AZURE_AD_TENANT_ID=your-tenant-id
NEXT_PUBLIC_REDIRECT_URI=/
```

Note that the `redirectUri` should match the one you configured in Azure AD.
Also, note that NEXT_PUBLIC env vars (browser env vars) are being applied only in during build time and not in runtime. Which means if these env vars are environment dependent, you should take it into account in your CI that handles the application build and deployment.

Next, Modify `_app.js` to Use MSAL Provider:

```javascript
import { MsalProvider } from '@azure/msal-react';
import { msalInstance } from '@/services/msal';
import '.@/styles/globals.css';

function MyApp({ Component, pageProps }) {
  return (
    <MsalProvider instance={msalInstance}>
      <Component {...pageProps} />
    </MsalProvider>
  );
}

export default MyApp;
```

### Adding Authentication to Your Pages

Here you can use MSAL React hooks which makes your life easier and add authentication functionality.

For example, in `pages/index.js`:

```javascript
import { AuthenticatedTemplate, UnauthenticatedTemplate, useMsal } from '@azure/msal-react';
import React from 'react';

function SignInButton() {
    const { instance } = useMsal();
    return <button onClick={() => instance.loginRedirect()}>Sign In</button>;
}

function WelcomeUser() {
    const { accounts } = useMsal();
    const username = accounts[0]?.name || 'Guest';
    return <p>Welcome, {username}</p>;
}

export default function Home() {
    return (
        <div>
            <AuthenticatedTemplate>
                <WelcomeUser />
            </AuthenticatedTemplate>
            <UnauthenticatedTemplate>
                <SignInButton />
            </UnauthenticatedTemplate>
        </div>
    );
}
```
Note that you can use the token acquired from MSAL to call your API with authentication header (if required) or use any other attribute from the token object such as name from the example.

You can also use the `useIsAuthenticated` hook to check if the user is authenticated and show different content accordingly. 

```javascript
import { useIsAuthenticated } from '@azure/msal-react';

export default function Home() {
    const isAuthenticated = useIsAuthenticated();
    return (
        <div>
            {isAuthenticated ? <WelcomeUser /> : <SignInButton />}
        </div>
    );
}
```

In one of my projects at work I created a custom hook to use the token in the API calls:

```javascript
import { useMsal } from '@azure/msal-react';

export const useAccessToken = () => {
  const { instance, accounts } = useMsal();
  const account = accounts[0];

  const getAccessToken = async () => {
    try {
      const request = {
        scopes: [`${process.env.NEXT_PUBLIC_CLIENT_ID}/.default`],
        account: account,
        redirectUri: '/',
      };

      const response = await instance.acquireTokenSilent(request);
      return response.accessToken;
    } catch (error) {
        console.log('error', error)
      if (error instanceof InteractionRequiredAuthError) {
        instance.acquireTokenRedirect(request);
      } else {
        throw error;
      }
    }
  };

  return getAccessToken;
};
```

And just for the sake of the example, one of components which uses the the custom hook which fetches the accessToken from AAD and sending as Authentication header in one if its API calls:

```javascript
import React, { useState, useEffect } from 'react';
import { useMsal } from '@azure/msal-react';

import { useAccessToken } from '@hooks/useToken';

const DeviceTable = () => {
  const getAccessToken = useAccessToken();
  const [regions, setRegions] = useState({});
  const [error, setError] = useState(null);

  useEffect(() => {
    let apiUrl = '';

    const fetchConfigAndData = async () => {
      try {
        const accessToken = await getAccessToken();

        const regionsData = await fetch(`${apiUrl}/api/v1/devices`, {
          headers: {
            'Authorization': `Bearer ${accessToken}`
          }
        }).then(res => res.json());
        setRegions(regionsData);
      } catch (error) {
        setError(error);
      }
    };

    fetchConfigAndData();
  }, []);
```

Keep in mind that i'm not an experienced frontend developer, so the code above might not be the best practice, but it works for me. I'll be happy to hear your feedback and suggestions. I'm mostly available on [Linkedin](https://www.linkedin.com/in/kazaz-or/).


Learn how to integrate Azure AD authentication into your Next.js app using MSAL library for security and user management.

All Blog Posts in the NextJS Category

Lab as a Service in DAZN

Integrating Azure AD Authentication in Your Next.js App Using MSAL