
We've implemented extensive improvements, including expanding device capacity, enhancing monitoring systems, introducing features like automated recovery mechanisms, chaging regions using internal network VPN and more.
These advancements will streamline testing processes, optimize resource utilization, and provide a seamless experience for developers and testers across global locations.
With ~250 users, supporting 25 different platforms and 150 testing devices, we are well-positioned for continued success.

For those of you who are not familiar with DAZN, here's a brief overview:

> *DAZN is the world’s leading live sports BTC streaming service. With DAZN, fans in any country can watch sport in any language, on any device, live and on-demand. Purpose built to stream live sports globally, DAZN is the market leader in Italy, Spain, Germany, and Japan, where it holds the top-tier domestic football rights — Serie A, La Liga, Bundesliga and J League, respectively — as well as the richest catalogue of live and on-demand content. DAZN is also available in over 200 other territories, where it boasts the most extensive library of elite and emerging sports content.*

DAZN develops both HTML-based and native applications designed to run on a variety of platforms, including TVs, set-top boxes, gaming consoles, and native devices. Ensuring high quality across these diverse platforms necessitates thorough testing at every stage of development, ranging from unit tests and component tests to emulator tests and real-device testing.

The global pandemic in 2020 prompted a paradigm shift in work practices, underscoring the importance of enabling engineers to access diverse resources and conduct testing remotely. Quality remains paramount, necessitating comprehensive testing methodologies despite remote work environments.

To address these challenges, we adopted a proactive “Lab as a Service” approach, culminating in the establishment of an in-house lab facility in Katowice, Poland. This strategic decision aimed to centralize resources and provide engineers with the necessary tools and infrastructure to conduct testing efficiently.

However, establishing and operating an in-house lab posed unique challenges that required careful consideration and resolution. These challenges encompassed logistical, technical, and organizational aspects, underscoring the need for adaptive solutions and collaborative efforts.

In response to these challenges, ongoing initiatives are focused on optimizing the efficiency and effectiveness of the in-house lab, streamlining workflows, enhancing collaboration among teams, and leveraging technology to overcome geographical barriers. By embracing innovation and agility, DAZN aims to continuously improve its testing capabilities and deliver exceptional quality across its diverse range of applications and platforms.

## Challenges

DAZN employs a diverse team of developers and test engineers spread across different global locations such as Poland, India, Israel, and the UK. Each team comes with its unique set of needs and requirements, necessitating a robust, highly available, and cost-effective platform to accommodate them effectively.

Key points include:

- Diverse Platform Support:
The platform must ensure comprehensive testing across all platforms supported by DAZN, emphasizing thoroughness and quality assurance in our testing procedures.
    
- Tailored Solutions:
Teams may require different tools and resources for their testing processes; for instance, some teams may rely on manual UI/UX testing devices, while others prioritize automation solutions.
    
- Integration Challenges:
Incorporating in-house tools and services into our platform is essential to ensure seamless continuity for engineers, requiring careful integration efforts.
    
- Scalability and Maintenance:
The platform needs to support a large number of engineers on a daily basis across multiple global sites, balancing scalability with efficient maintenance to avoid operational bottlenecks.

- Cost Considerations:
Maintaining devices, infrastructure, and manpower across various locations can incur significant expenses, necessitating a cost-effective approach to resource allocation and management.

- Ownership and Accountability:
Clear ownership and accountability structures are crucial to prevent issues such as maintenance gaps or device mismanagement.

To address these challenges, ongoing efforts are focused on streamlining processes, enhancing collaboration between teams, and leveraging automation wherever possible to optimize resource utilization and improve efficiency in our testing and development workflows.


## Solution — Lab as a Service

Our team, composed of seasoned experts in DevOps, Automation Infrastructure, and IT, recognized the need for a novel solution to address the challenges faced in managing Device Under Test (DUT) usage at DAZN. In the first quarter of 2023, we conducted a comprehensive assessment to identify and map out the major pain points associated with DUT utilization within the organization. This assessment culminated in the development of a pioneering approach coined “Lab as a Service,” aimed at alleviating the burden on developers and testers by handling all DUT-related tasks and physical infrastructure management.

During the second quarter of 2023, we introduced the Minimum Viable Product (MVP) setup, leveraging the Suitest framework — a vendor offering remote control and automation capabilities tailored for both HTML-based and native platforms. Our team meticulously orchestrated the deployment of all requisite physical infrastructure, assembling a dedicated lab housing 50 test devices within a spacious room at the DAZN Katowice office.

![DUT lab v1.0 (QA room — May 2023)](https://miro.medium.com/v2/resize:fit:640/format:webp/1*MDbJo7nq77qwWrRCpNvsrA.jpeg)*DUT lab v1.0 (QA room — May 2023)*

To bolster reliability and resilience, we implemented a comprehensive monitoring and alerting system, with Grafana serving as the frontend interface. Additionally, we integrated DUT recovery mechanisms to enhance the robustness of our testing environment.

![Grafana monitoring dashboard](https://miro.medium.com/v2/resize:fit:640/format:webp/1*8a9dpHjMLH1FZCQptCNplw.png)*Grafana monitoring dashboard*

The efficacy of these enhancements became apparent swiftly, as an increasing number of engineers gravitated towards utilizing the centralized lab infrastructure instead of configuring local devices. Leveraging the valuable feedback provided by users, we iteratively refined and enhanced the lab environment, ensuring that it continues to meet the evolving needs of our development and testing teams.

## Lab as a service 2.0

As the user base and the number of devices in the DAZN lab continued to surge, spurred by the establishment of a new DAZN site in India and the expansion of supported platforms throughout the latter half of 2023, we undertook further enhancements to the Katowice lab. These adjustments were aimed at accommodating the organization’s growth and fortifying its capacity to scale even further. Throughout the first quarter, we initiated a series of improvements targeting both the physical infrastructure and supporting features:

### Infrastructure enhancements

- __Relocation and Expansion:__

The DUT lab underwent relocation to a larger space, facilitating increased scalability and the seamless onboarding of additional platforms and devices. Leveraging existing racks more efficiently, we enabled the mounting of devices on both sides of each rack, resulting in a 50% boost in the number of TVs and a 40% increase in the number of set-top boxes (STBs), totaling 120 DUTs overall, with ample room for further expansion.

![DUT lab v2.0 (open space — January 2024)](https://miro.medium.com/v2/resize:fit:640/format:webp/1*IKUuw6aU2wQMzTBKCKS_bA.jpeg)*DUT lab v2.0 (open space — January 2024)*

- __Streaming Solutions:__

Collaborating with Suitest, we identified a cost-effective solution enabling the streaming of device HDMI output for STBs and gaming consoles. Additionally, we enhanced the TV streaming setup by refining the phone streaming app.

![Live DUT Streaming Phone camera setup](https://miro.medium.com/v2/resize:fit:720/format:webp/1*K460jjXRPqXUVQf8PVg0ng.jpeg)*Live DUT streaming mobile phone camera setup*

![HDMI grabber/splitter combo](https://miro.medium.com/v2/resize:fit:640/format:webp/1*s-cKLE8Oh1h5mliovKc91Q.jpeg)*HDMI grabber/splitter combo*

### Features enhancements

- __Collaboration with Suitest:__

Over the course of the year, we actively collaborated with Suitest to influence their roadmap, contributing to improvements in user experience and interface. A significant milestone was the introduction of a Unified UI integrating video streaming, remote control, and logs within the same browser window, enhancing productivity for lab users.

![Suitest Unified UI](https://miro.medium.com/v2/resize:fit:640/format:webp/1*Wg96k859tBYZwz13vVXG3Q.png)*Suitest Unified UI*

- __LR (Living Room) Launcher__:

In collaboration with the DAZN LivingRoom development team, we introduced LR Launcher, simplifying the workflow for manual testers and streamlining the onboarding process for partner platforms such as Sky and Movistar for the sake of ease of use when testing different development environments.


![LR Launcher](https://miro.medium.com/v2/resize:fit:640/format:webp/1*5AObQs1_Vdyd9cXhf5qF3w.png)*LR Launcher*

- __Enhanced Monitoring__:

With the growing user base and device count, robust monitoring became paramount. We revamped our monitoring service, streamlining processes and transitioning from an API-based solution to WSS notifications. Furthermore, we expanded monitoring coverage and implemented alerting for control units and streaming devices.

- __Automated Recovery__:

Leveraging smart plugs across all TV platforms, we implemented remote manual and automated recovery mechanisms for malfunctioning DUTs. Efforts are underway to develop similar mechanisms for cameras and control units, bolstering the resilience and efficiency of our lab environment.

- __Local Tunnel__:

A local tunnel solution providing exposure of a local development server allows developers and testers to seamlessly execute their code on a real physical lab device. This capability not only streamlines the testing process but also provides a practical and authentic environment for validating local code during development.

- __Lab Insights Report (BI)__:

An automated report detailing lab usage and testing trends. This report will provide valuable insights into usage patterns, enabling us to further optimize our services based on real data.

![BI Report](https://res.cloudinary.com/dtt2usukl/image/upload/v1716458627/Image_27-08-2023_at_11.58_sas9dh.jpg)*BI Report*

__VPN Change Region & VPN Console__:

A VPN solution that will empower lab users to seamlessly transfer DUTs between regions remotely. Using firewalls controlled by OPNSense, a lambda backend and a UI portal written in React (NextJS) users are able to testing of region-dependent content across all lab devices by switching regions as needed.

![VPN Console](https://res.cloudinary.com/dtt2usukl/image/upload/v1716458833/Image_23-05-2024_at_13.06_oxbech.jpg)*VPN Console*

## Service & Support

- __Continuous Improvement:__

Since the launch of the DUT lab in May 2023, we have continually bolstered our service and support infrastructure. This includes expanding our support engineering team, enhancing documentation and guides, and maintaining accessible support channels during working hours. Our commitment to swift issue resolution is underscored by our expanded remote resolution capabilities, facilitated by proactive monitoring and device error-based recovery processes.

- __Onboarding Optimization:__

Through the introduction of “Lab as a Service 2.0” and tailored training sessions, we have streamlined the onboarding process for new users, ensuring a seamless transition and maximizing productivity from the outset using official docs portal (using github pages and backstage).

These enhancements collectively represent our ongoing commitment to optimizing the DAZN lab environment, fostering innovation, and empowering our engineering teams to deliver exceptional quality across all platforms and devices.


## Acknowledgements

Special thanks to the amazing AP Nexus team for their dedication and hard work, it was a pleasure working with each one of you!

[Dennis Groisman](https://www.linkedin.com/in/dennis-groisman-9811b335/), [Daniel-Shahar Cohen](https://www.linkedin.com/in/danielshaharcohen/), [Rafal Muszalik](https://www.linkedin.com/in/rafa%C5%82muszalik/), [Blazej Milewski](https://www.linkedin.com/in/blazej-milewski/), [Alex Rivkin](https://www.linkedin.com/in/alex-rivkin-b037a6a1/), [Yoav Brown](https://www.linkedin.com/in/yoav-brown-970b55265/) and the one and only [Gal Moran](https://www.linkedin.com/in/gal-moran-a3454095/).


[![DAZN Lab as a Service](https://res.cloudinary.com/dtt2usukl/image/upload/v1716460903/play_video_lab_fnkajn.jpg)](https://res.cloudinary.com/dtt2usukl/video/upload/v1716460249/tvlab_v2_withmusic_twifar.mp4 "DAZN Lab as a Service - Nexus Team")*My team - Lab as a Service 2.0*


Article on [Medium (DAZN Engineering Blog)](https://medium.com/dazn-tech/lab-as-a-service-in-dazn-9bcc9d549970)


I'm incredibly proud to share the outstanding work our team has accomplished recently in upgrading our testing lab infrastructure.

Lab as a Service in DAZN


Every web developer, either frontend or backend, may have come across this CORS policy violation (Cross-Origin Resource Sharing) error message at least once in their career.

The error itself is actually pretty informative. It basically tells you that the client-side is not one of the "whitelisted" origins to access the data being fetched. This is a security mechanism that browsers implement to prevent malicious users from accessing the data from a different origin.

Let's dive in a little bit.

First, what is CORS exactly? CORS stands for Cross-Origin Resource Sharing, which is a security mechanism that allows/prevents one origin to access a resource from a different origin. This is something that the server has control over to restrict who has access to the resource, how they can access the data, for example, which HTTP methods, HTTP headers, etc are allowed.
At its core, you can treat it as a protocol allowing web applications to request resources from different domains securely. It's a dance of headers and requests, where the server dictates who can access its resources.


__Same Origin Policy vs Cross Origin Resource Sharing__

The Same-Origin Policy (SOP) is a critical security mechanism that restricts web pages from making requests to a different origin than the one that served the web page. CORS, on the other hand, is a tool to relax those restrictions under controlled conditions. It allows servers to specify who can access their assets and how.

__CORS Mechanism__

CORS works by adding new HTTP headers that allow servers to declare which origins are permitted to access the resources. For example, the `Access-Control-Allow-Origin` header can specify an allowed origin, or it can use `*` to allow requests from any origin. However, using `*` in most cases is discouraged for sensitive resources due to security concerns.



Let's take a look at an example using NodeJS & Express:

```javascript
const express = require('express');
const cors = require('cors');
const app = express();

app.use(cors());

// CORS with specific options
const corsOptions = {
  origin: 'https://example.com',
};

app.get('/data', cors(corsOptions), (req, res) => {
  res.json({ message: 'This route is CORS-enabled for an allowed origin.' });
});

app.listen(3000, () => console.log('Server running on port 3000'));
```

Here we're basically using the `cors` middleware that is applied globally to allow Cross-Origin Resource Sharing (CORS) from any origin, enabling client applications from different domains to access the server. Additionally, it showcases a more complex setup where CORS settings are customized for a specific route, allowing requests only from a specified origin and setting specific methods and headers.

Now, let's take a look at another example, but this time using Python & FastAPI:

```python
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware

app = FastAPI()

# Simple CORS
app.add_middleware(
    CORSMiddleware,
    allow_origins=["*"],
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

# CORS for specific origins
origins = [
    "http://example.com",
]
app.add_middleware(
    CORSMiddleware,
    allow_origins=origins,
    allow_credentials=True,
    allow_methods=["GET"],
    allow_headers=["X-Custom-Header"],
)

@app.get("/data")
async def get_data():
    return {"message": "This route is CORS-enabled for specific origins."}
```

This simple snippet demonstrates two approaches: a global application of CORS allowing all origins, methods, and headers, and a more restrictive approach where CORS is configured for specific origins, methods, and headers on all routes, illustrating how to control access to your API more granularly.

Implementing CORS correctly is a balancing act. Developers must ensure their web applications are accessible to legitimate users and services while protecting against unauthorized access. Understanding CORS is crucial for any developer working on the web, as it directly impacts the security and functionality of web applications.


I can assure you that one of the first issues you'll encounter when testing your newly developed product will be CORS. So what is CORS exactly?

CORS - Introduction For The New FE/BE Developer


Makefiles are powerful tools in a developer's arsenal, capable of significantly improving efficiency and consistency in the build process. By understanding and utilizing Makefiles for Docker and Terraform, developers can streamline their workflows, reduce repetitive tasks, and focus more on the creative aspects of software development.

In this article, we'll explore the basics of Makefiles and how they can be used to automate the build process with examples of Docker and Terraform commands usage.

### What is a Makefile?

Incorporating a Makefile into your project can greatly streamline various development tasks, such as managing dependencies, running tests, building documentation,formatting code, performing static analysis, build/compile/transpile, cleaning up temporary files and more.

The flexibility of Makefiles allows you to automate routine processes in a structured and efficient manner. Here's a more detailed look at how you can leverage Makefiles in your projects:


### Basic Structure of a Makefile

A Makefile typically consists of targets, prerequisites, and recipes. Targets represent the goals you want to achieve, which could be actions like running your app or installing dependencies. Prerequisites are files or other targets needed before a target can be executed. Recipes are shell commands that make will execute.

For instance, you might have a `run` target to execute your Python application, a `setup` target for installing dependencies listed in `requirements.txt`, and a `clean` target for removing Python bytecode files or the `__pycache__` directory.

### Advanced Techniques

__Default Goals and Help Messages:__ Setting a default goal, like `.DEFAULT_GOAL := help`, allows you to display a helpful message when make is run without a target.

__Virtual Environments:__ Integrating virtual environments in your Makefile ensures that the right Python environment and dependencies are used. This can be crucial in projects with specific version requirements or multiple dependencies.

__Injecting Paths into `PYTHONPATH:`__ This advanced trick allows Python to search for packages in specified file system paths without needing to install them, which can be particularly useful in projects with heavy dependencies or multiple branches.


### Some Practical Examples

__Running Tests with Coverage:__ You can create a coverage target to run tests with coverage, specifying the commands for running the tests and generating a coverage report.

__Linting and Static Analysis:__ A lint target can be set up to run various linting and static analysis tools like flake8, pylint, and mypy on your code.

__Task Dependencies and Parameters:__ Makefiles allow you to specify dependencies between tasks. For example, a test target might depend on a lint target. You can also pass parameters to tasks, like specifying an IP and port for a serve task.

__Makefile for Docker Commands:__ 

By leveraging Makefiles in your Python projects, you not only save time but also create a more robust and maintainable development environment. These examples and advanced features illustrate just how versatile and powerful Makefiles can be in automating and streamlining your Python development workflow.

```makefile
export TOP := $(shell git rev-parse --show-toplevel)
export IMAGE_NAME := my-docker-image
export CONTAINER_NAME := my-container-image

build:
	docker build --build-arg JFROG_USERNAME=${JFROG_USERNAME} --build-arg JFROG_ARTIFACTORY_TOKEN=${JFROG_ARTIFACTORY_TOKEN} -t $(IMAGE_NAME) .

run:
	$(TOP)/scripts/run_docker.sh &

exec:
	docker exec -it $(CONTAINER_NAME) /bin/sh

delete:
	docker stop $(CONTAINER_NAME)
	docker rm $(CONTAINER_NAME)
```
In this example the `build` command creates a Docker image with build arguments (`make build`).

The `run` command runs a script to start the container in the background (`make run`). It runs a bash script that starts the container in the background and saves the container ID in a file.

The `exec` provides an interactive shell within the container for debugging proposes (`make exec`).

The `delete` stops and removes the container (`make delete`).

__Makefile for Terraform Commands:__

Terraform, an infrastructure as code software tool, also benefits from Makefiles, its commands can become quite complex. Here's an example on why it is useful to use Makefiles for Terraform:

```makefile
account := dev
namespace ?= $(shell echo $(USER) | awk '{print tolower($0)}')
service ?= $(shell basename `git rev-parse --show-toplevel`)

ifeq ($(account), dev)
    AWS_PROFILE := 123456
else ifeq ($(account), stage)
    AWS_PROFILE := 654321
else
    $(error Invalid account specified: $(account))
endif

init:
	terraform init -reconfigure \
		-backend-config=config/$(account)/backend.tfvars -backend-config=key=services/$(namespace)/$(service).tfstate

plan:
	terraform plan -out /tmp/$(service).tfplan \
		-var-file=config/$(account)/terraform.tfvars -var-file=config/$(account)/service.tfvars \
		-var "namespace=$(namespace)" -var "service_name=$(service)"

apply:
	terraform apply /tmp/$(service).tfplan

validate:
	@terraform validate

destroy:
	terraform destroy \
		-var-file=config/$(account)/terraform.tfvars -var-file=config/$(account)/service.tfvars \
		-var "namespace=$(namespace)" -var "service_name=$(service)"
```

This Makefile automates various Terraform commands like `init` which initializes a new or existing Terraform configuration (`make init`).
This command is particularly used to initialize the working directory containing Terraform configuration files, with -reconfigure to force re-initialization. It specifies backend configuration variables and the state file location.

The `plan` command creates an execution plan. This command creates a Terraform plan and outputs it to a temporary file. It specifies variable files and sets additional variables like namespace and service_name (`make plan`).

The `apply` command applies the changes required to reach the desired state of the configuration (`make apply`).

The `validate` command is used to validate the syntax and configuration of the Terraform files (`make validate`).

The `destroy` command is used to destroy all resources managed by the current Terraform configuration (`make destroy`).

In conclusion, Makefiles stand as a fundamental and powerful tool in software development, offering a versatile approach to automating and streamlining various aspects of the build process. Whether you're working with Docker containers or managing infrastructure with Terraform, Makefiles bring a level of efficiency and consistency that can significantly enhance your workflow. By effectively leveraging Makefiles, developers not only save valuable time but also ensure a more robust and maintainable development process, making them an essential component in any modern software project.


Makefiles are a cornerstone tool in the software development process, particularly useful for streamlining and automating the build process.

Why Makefile is One of My Essentials Tools in Every Software Project


At work, I manage a repository for a front-end application, which includes Markdown files that are compiled into a documentation portal.

Documentation, especially in Markdown format, is more than just words on a page. In my case, it's a user's guide, a developer's reference, and sometimes, even a product's first impression. Therefore, ensuring its quality, consistency, and accuracy is critical, and this is where Markdown linters come into play.

### Essential Tools for Markdown Linting I Used

Our approach involves three key tools, each serving a unique purpose:

__Markdownlint:__ For stylistic consistency and syntax correctness.

__Markdown-link-check:__ To validate all hyperlinks are functional.

__cspell__ (Code Spell Checker): For ensuring correct spelling throughout your documentation.

In my case, we used to run these linters on each pull request/commit that was uploaded to the repository.

### Linters

__Markdownlint:__

```bash
npm install markdownlint-cli
```

Ensures Markdown files follow best practices and maintain a consistent style. Of course, as anything the following different rules, it can be customized to your needs.

Markdown lint follows its defined rules, you can find it [here](https://github.com/markdownlint/markdownlint/blob/main/docs/RULES.md).

To modify it, you can use a configuration file, `.markdownlint.json`, which can be placed in the root of your repository.

For example:

```json
{
    "MD001": false,
    "MD013": false,
    "MD028": false,
    "MD029": false,
    "MD036": false,
    "MD033": {
        "allowed_elements": [
            "iframe",
            "url"
        ]
    }
}
```

This config file ignore certain rules defined by the tool, and allows specific elements that are not allowed by default.

__Markdown-link-check:__

```bash
npm install markdown-link-check
```

Checks for broken links in your Markdown files, ensuring all external references are valid. Note that this tools fails to check internal links from obvious reasons, therefore you can configure it to ignore them.

You can use `mlc_config.json` file to configure the tool, and place it in the root of your repository.

```json
{
    "ignorePatterns": [
        { 
            "pattern": "^https://github.com/myorganization/" 
        }
    ]
}
```

myorganization is the name of your organization in GitHub in this example.

__cspell:__

```bash
npm install cspell
```

Catches spelling mistakes, a critical aspect of maintaining professional and clear documentation. Although sometimes you might have words which aren't really a typo, but are not in the dictionary, therefore you can add them to the dictionary.

You can use `cspell.json` file to configure the tool, and place it in the root of your repository.

```json
{
    "words": [
        "tizen",
        "webos",
    ],
    "ignoreRegExpList": [
        "\\!\\[[^\\]]*\\]\$[^)]*\$",
        "(?<=```[\\s\\S]*?)\\S+",
        "\\[.*?\\]\$https?://[^\$]*\\)"
    ]
}
```

Note that I also added regex expression to ignore certain patterns, such as links, code blocks, and images alt text.


### Setting Up GitHub Actions Workflow

Integrating these tools into a GitHub Actions workflow ensures that every pull request adheres to your documentation standards. Here’s how you can configure the workflow for each tool:

```yaml
name: Documentation Checks

on:
  pull_request:
    paths:
      - 'docs'
      - 'docs/**'

jobs:
  markdown-lint:
    runs-on: ubuntu-latest
    timeout-minutes: 5

    steps:
    - name: Checkout
      uses: actions/checkout@v3
      with:
        fetch-depth: 0
  
    - name: Setup-env
      uses: ./.github/actions/setup-env
      with:
        node-version: '18.16'

    - name: Run markdownlint
      run: npm run lint

  link-checker:
    runs-on: ubuntu-latest
    timeout-minutes: 5

    steps:
    - name: Checkout
      uses: actions/checkout@v3
      with:
        fetch-depth: 0

    - name: Setup-env
      uses: ./.github/actions/setup-env
      with:
        node-version: '18.16'

    - name: Run markdown-link-check
      run: npm run link-check

  spell-checking:
    runs-on: ubuntu-latest
    timeout-minutes: 5

    steps:
    - name: Checkout
      uses: actions/checkout@v3
      with:
        fetch-depth: 0

    - name: Setup-env
      uses: ./.github/actions/setup-env
      with:
        node-version: '18.16'

    - name: Run cspell
      run: npm run spell-check
```

This is a simple workflow that runs on every pull request, and runs each linter separately.
In this case I specified the paths to the docs folder, but you can change it to your needs.

In the beginning of the workflow, I used a very simple composite github action to install node and npm, and then run the linters so i won't have to duplicate the same steps in each job. It's a simple action, but it's very useful, you can improve it to your needs.

If your'e interested in the composite action, here it is:

```yaml
name: 'Setup Environment'
description: 'Checkout and setup Node.js'
inputs:
  node-version:
    description: 'NodeJS version'
    required: true
    default: '18.16'

runs:
  using: "composite"
  steps:

    - name: Setup Node
      uses: actions/setup-node@v3
      with:
        node-version: ${{ inputs.node-version }}
    
    - name: install dependencies
      shell: bash
      run: npm install
```

![markdown-linters-workflow](/images/markdown-workflows.jpeg)

By integrating these Markdown linters into your GitHub Actions workflow, you elevate the standard of your project's documentation. This setup not only ensures consistency and correctness but also encourages a culture of quality and attention to detail within your development team. Remember, good documentation reflects the quality of your code and your commitment as a developer.


Do you have a repo that contains important documentation? Streamline your documentation with Markdown linters to enhance documentation quality.

Markdown Linters for Your Docs Repo


I encountered this task at work where I needed to monitor a local machines that are considered critical for us in terms of tests runners and CI. 
The machines were running a few services that required some effective monitoring and alerting.

Recently, our team has transitioned from NewRelic to Coralogix for monitoring, and integrating it with OpenTelemetry Collector Contrib was done for all of our local machines. This post details the steps involved.

After a short research, I found out that OpenTelemetry Collector Contrib has a [hostmetrics receiver] that can be also used to monitor local machines and send the data directly to our new vendor - Coralogix.

OpenTelemetry provides a single set of APIs and libraries that standardize how you collect and transfer telemetry data. You can use these APIs and libraries to instrument your code and send telemetry data to a backend of your choice. The OpenTelemetry Collector offers a vendor-agnostic implementation on how to receive, process, and export telemetry data. In addition, it removes the need to run, operate, and maintain multiple agents/collectors in order to support open-source observability data formats (e.g. Prometheus, etc.) sending to multiple open-source or commercial back-ends.

__So let's start__

Download the otelcol-contrib binary for your platform from OpenTelemetry [GitHub](https://github.com/open-telemetry/opentelemetry-collector-releases/releases).

If you're using Coralogix as well, you'll need to obtain your Coralogix Private Key and set it along with other required environment variables:

```bash
CORALOGIX_DOMAIN
CORALOGIX_APP_NAME
CORALOGIX_SUBSYS_NAME
CORALOGIX_PRIVATE_KEY
```
Create a `config.yaml` file, including necessary metrics, scrapers, log level, and intervals. Refer to [OpenTelemetry's Host Metrics Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/receiver/hostmetricsreceiver) for further information.

Add coralogix as an exporter in the config, as shown below as an example:

```yaml
receivers:
  hostmetrics:
    collection_interval: 30s
    scrapers:
      network:
      cpu:
      memory:
      disk:
exporters:
  coralogix:
    traces:
      endpoint: "otel-traces.${CORALOGIX_DOMAIN}:443"
    metrics:
      endpoint: "otel-metrics.${CORALOGIX_DOMAIN}:443"
    logs:
      endpoint: "otel-logs.${CORALOGIX_DOMAIN}:443"
    domain: "$CORALOGIX_DOMAIN"
    private_key: "$CORALOGIX_PRIVATE_KEY"
    application_name: "$CORALOGIX_APP_NAME"
    subsystem_name: "$CORALOGIX_SUBSYS_NAME"
    application_name_attributes:
      - "service.namespace"
    subsystem_name_attributes:
      - "log.file.name"
      - "service.name"
    timeout: 30s
  logging:
    verbosity: normal
service:
  pipelines:
    metrics:
      receivers: 
       - hostmetrics
      exporters: 
       - coralogix
```

Execute the binary with the config file flag: 

```bash
./otelcol-contrib --config=config.yaml
```

And there you go. You should be able to see the metrics in your metrics vendor exploration by the subsystem name or application name.

![Coralogix-Widget](/images/coralogix-widget.png)


Learn how to seamlessly integrate OpenTelemetry Collector Contrib with Coralogix for efficient monitoring of local hosted machines.

Monitor Your Hosted Local Machines with OpenTelemetry Collector Contrib


I'll start with a disclaimer - I'm far from a Linux expert. But Linux fascinates me, and i'm using a Linux based OS on a daily basis. That's why i'm not writing this blog post from an expert point of view, but rather from a curious one.

I first encountered the Nice term in Linux in my previous work place where we worked on Embedded devices running on ARM architecture with Buildroot as a distribution. We had a bug where one of the processes failed to load on time and we eventually fixed it by tweaking its niceness value (that's in a very high level, but you get the idea).

To understand what is niceness, we need to understand what is process priority and the Linux scheduler first.

Every operating system has a scheduler. The job of the scheduler is to allocate processor time to the various processes that are running in parallel. The scheduler must make decisions about which process to run next and for how long, especially when there are multiple processes requesting CPU time at the same time.

This is where priorities come into play. In Linux, every process is assigned a priority number. The lower this number, the higher the priority. The scheduler uses these numbers to decide which process to run next.

So How is nice related to priority?

Niceness is an indirect representation of priority. Instead of dealing with priorities directly, Linux uses the concept of niceness to allow users to influence the scheduler's decisions in a more user-friendly manner.

Niceness values range from -20 (highest priority) to 19 (lowest priority).
The actual priority value (often referred to as the "nice value") ranges from 1 (lowest priority) to 40 (highest priority).
When a process has a niceness value of 0, its actual priority is 20. A niceness value of -20 would adjust the priority to 0, while a niceness value of 19 would adjust it to 39.

A process with a high niceness value (close to 19) is "nicer" to other processes, implying it's less demanding of CPU time. Conversely, a process with a low niceness value (close to -20) is less "nice," indicating it's more demanding.

![niceness-scale](/images/niceness-scale.png)

For example, let’s say with run a command like `ls`, in most cases it’ll get the 0 niceness priority (default) as all other process unless it was defined differently.
You could, for the sake of the example, run `nice ls` and it’ll result with a niceness level of 10.

Now Let’s say you don’t want to use defaults, you could run `nice -n 15 ls` and it’ll will run with a niceness level of 15.

```bash
$ ps -l
F S   UID   PID  PPID  C PRI  NI ADDR SZ WCHAN  TTY          TIME CMD
0 S  1000 15044 15035  0  80   0 -  7453 wait   pts/29   00:00:00 bash
0 S  1000 15052 15044  0  60   15 -  3976 hrtime pts/29   00:00:00 ls
0 R  1000 15080 15044  0  80   0 -  4680 -      pts/29   00:00:00 ps
```

The whole point of niceness is to not allow normal users to set high priorities for their process. Lets say for example I want to run with high priority `nice -n -20 ls`, you’ll usually get an output like this:

```bash
nice: cannot set niceness: Permission denied
```
And the command will run with default niceness level (in most common cases).
This is because such operation is reserved for root users so normal users won't break their systems.


In practical terms, when we should use which niceness level? Here are some examples:

- A CPU-intensive task that you don't need to finish immediately (like a background data processing job) might be assigned a high niceness value to ensure it doesn't starve other processes of CPU time.
- A real-time task that needs to execute immediately (like audio or video processing) might be given a low niceness value to ensure it gets as much CPU time as it needs.

__Adjusting Niceness__

Suppose you're running a large compilation job, a backup, or any task that can potentially hog all the CPU like dealing with large files. You might want to run it with a high niceness value to prevent it from slowing down other processes.

```bash
nice -n 19 tar -czf backup.tar.gz /home/user
```

For tasks that require immediate processing, like audio/video encoding or real-time simulations, you might want to decrease their niceness value.

```bash
sudo nice -n -10 ffmpeg -i input.mp4 output.mp3
```

Let's say you started a process without modifying its niceness but later realized it's consuming too much CPU. You can dynamically adjust its niceness using `renice` and the process PID (which you can get with a simple `ps -l` command).

```bash
renice -n 15 -p [PID]
```

In the Linux ecosystem, understanding process scheduling, priority, and niceness is invaluable. By tweaking these parameters, software developers can optimize applications for performance, responsiveness, and resource usage. Whether it's ensuring smooth multimedia playback, prioritizing time-sensitive tasks, or preventing resource-intensive processes from overwhelming the system, the power of niceness offers developers a robust tool in system optimization.


Get to know the Linux process priority and Niceness.

Linux Process Prioritization with Niceness


I came across pre-commit hooks a while ago and I was amazed by how easy it is to use and how much it can improve your git workflow. I got to see how useful it can be especially when working on large project or when lots of developers are working on a repository.

The good thing about pre-commit, is that it’s quite straightforward to setup. To showcase how easy it is, I’ll use mypy as an example, as this is a real pre-commit hook I had to setup on one of our repositories.

Just a refresher - mypy is a static type checker for Python. It is used to find bugs mostly related to types in your code without actually running it. For more information about mypy you can checkout my previous blog post about it - [mypy](/blogs/mypy-integration)

To set up a pre-commit hook that will run mypy before allowing a commit, you can use the pre-commit library, which is a tool (or a framework actually) for managing and maintaining multi-language pre-commit hooks. Of course, you can use it for every check you like such as linting, testing, etc.

Here are the steps to set it up:

1. Install the pre-commit framework. If you haven't already installed it, you can do so with pip/poetry:
```bash
pip install pre-commit # pip
```
```bash
poetry add pre-commit@latest --group dev # poetry
```

2. Create a `.pre-commit-config.yaml` file. This file is used to configure what actions pre-commit will take. The .pre-commit-config.yaml file is typically placed at the root of your repository as pre-commit tool will look for this file in the root directory by default when it runs.

Here's an example of what it might look like for running mypy:

```yaml
repos:
-   repo: https://github.com/pre-commit/mirrors-mypy
    rev: v1.4.1
    hooks:
    -   id: mypy
        args: ["--config", "mypy.ini"]
        files: ^src/
```

This configuration tells pre-commit to use the mypy pre-commit hook from the mirrors-mypy repository.
The rev field should be set to the version of mypy you want to use, while the args are the CLI arguments you want to it to run with, in the case as exampled above, i’m just passing mypy’s config file that already includes all the desired flags.

3. Install the git hook scripts. You only need to do this once after cloning the repository.
You can do this by running the following command in your git repository:

```bash
pre-commit install
```

This will install a pre-commit script in your ``.git/hooks/`` directory that will automatically run your configured hooks when you run git commit.

4. Test the pre-commit hook. You can test the hook by trying to make a commit. If mypy finds any type errors, the commit will be blocked.


Couple of additional notes:

- Keep in mind, by default, pre-commit runs only on files that are staged for commit, but you can configure it to run on all the project’s file if you’d like just by telling it to check all .py files for example.

- Once someone clones the repository, in order to contribute, running the install command of pre-commit is a must. I'm aware there are some methods available to do this automatically, but i haven't seen one that I like.

- Uploading the ``.git/hooks`` directory into github is not recommended of course.

- If you wish to place your .pre-commit-config.yaml file elsewhere which is not the root directory, such as in a .github/pre-commit/ directory, you can do so by specifying the `PRE_COMMIT_CONFIG_FILE` environment variable to point to its location.
For instance, if your .pre-commit-config.yaml is in .github/pre-commit/, you can set PRE_COMMIT_CONFIG_FILE as follows:
``export PRE_COMMIT_CONFIG_FILE=.github/pre-commit/.pre-commit-config.yaml``

Keep in mind that anyone cloning your repository will also need to set this environment variable, or pre-commit will not be able to find the configuration file. Due to this potential complication, it is usually easier to keep the .pre-commit-config.yaml at the root of your repository.


Discover how pre-commit hooks streamline workflows, ensure code quality, and simplify collaborations using a real-world example.

Leverage Pre-Commit Hooks to Improve Your Git Workflow


While Git provides a suite of powerful tools out of the box, the true strength of this version control system lies in its flexibility and adaptability. In this blog post, we will dive into a specific aspect of Git that often goes underappreciated: customizing the git log command. By customizing your git log output, you can enable yourself to make your Git logs look exactly the way you want.

Before we get into it, let's understand the default behavior of `git log` command. By default, git log shows the commit hash, author, date, and commit message. While this is helpful, you can modify it to show more specific and visually appealing details. How?

You can do this by setting aliases in the The `.gitconfig` config file. This is where you can define global configurations and aliases.

So in order to show case some of the stuff can be done as aliases, i'll share some of my aliases the resides in my gitconfig file:

__git lg__

```bash
[alias]
    lg = log --color --graph --pretty=format:'%C(yellow)%h%Creset -%C(red)%d%Creset %s %Cgreen(%cr) %C(bold blue)<%an>%Creset' --abbrev-commit
```
This alias provides a colorful, abbreviated commit log with a graph on the side.

![git-lg](/images/git-lg.png)

__git lol__

```bash
[alias]
    lol = log --graph --decorate --pretty=oneline --abbrev-commit --format=format:'%C(yellow)%h%C(reset)%C(red)%d%C(reset) %C(white)%s%C(reset) %C(green)(%ar)%C(reset) %C(bold blue)<%an>%C(reset)'
```

Building on the previous alias, this one presents an even more compact one-liner version of the log.

__git lola__

```bash
[alias]
    lola = !git lol --all
```
Extends the lol alias by displaying logs from all branches.

__git lgc__

```bash
[alias]
    lgc = !git lg | sed '$a\\' | grep '* ' | tac | cat -n | sed 's/^[ \t]//' | sed 's/[\t][\\*]//' | tac | less -XR
```
This is a more advanced alias that manipulates the output using sed and less to display the log entries in a structured manner with an accompanying index.

----
Looking at the lg and lol aliases, there are several formatting options used:

___--graph:___ Displays an ASCII graph of the branch and merge history.

___--pretty=format:'...':___ This option allows us to specify the format of the log output. Within the format string, we can use placeholders like %h (abbreviated commit hash), %an (author name), and %s (commit subject).

___--abbrev-commit:___ This shortens the commit hash to a smaller, more readable length.

___Color Formatting:___ Using %C(color_name) before a placeholder will change its color in the log. For instance, %C(yellow)%h%Creset will display the commit hash in yellow.

To conclude this short one, customizing Git logs is more than just a visual preference. It can drastically enhance your Git experience, helping you quickly grasp your repository's history and the relationships between commits and branches. By using the `.gitconfig` to its full potential, you can mold Git's behavior to suit your unique workflow and aesthetic preferences.


By customizing your git log output, you can make your Git logs look exactly the way you want which could make your life easier.

Customizing Your Git Log Output


As my career in the tech industry grew in the QA discipline, I always had a hidden passion for coding. I've always been a "techy" person, and I've always been interested in programming. I had a very little experience as a kid who liked building websites and at later stages I fully adopt Linux OS's. However, I never had the courage to take the leap and start learning. Whenever I got the chance to do it, I sat in front of my empty editor and felt like I didn't know where to start.

This is a good point to make a disclaimer, in the companies I worked as a QA and as a Team lead, I always worked closely to developers, and I had got to a point i'm comfortable with Python (not for actual coding, but more in terms of not being intimidated by it).

At some point I got the opportunity to help a small automation team we had in a startup I worked in, and as time went by, I got more and more involved in the code, and I started to feel more comfortable with it up until the point I was able to do some contributions myself mainly around end-to-end tests we had for our backend services.

After some period of combining automation tests and management, the startup company I worked in got acquired by a bigger company, and I got promoted to a higher managerial role that turned the tied for me. It wasn't until I got promoted to a managerial role that drew me away from the fun parts of working in tech, that I realized I need to do something about it. I was no longer involved in the code, and I felt like I was missing out on the fun parts of working in tech. I decided to take action and start learning to code.

Then I encountered these hashtags on Twitter of #100DaysOfCode. I though the concept was amazing, but I wasn't sure where to start, until I found the [100DaysOfCode Udemy Python Course by Angela Yu](https://www.udemy.com/course/100-days-of-code/).

So I went for it, and what can I tell you? It was totally worth it. You'll learn the fundamentals of Python, and you'll get a good understanding of the language. You'll also get to know common libraries such as tkinter, Selenium, Beautiful Soup, Request, Flask, Pandas, NumPy, Scikit-Learn, Plotly, Matplotlib and much more. note I said get to know, as it high level.
Angela is an amazing instructor, and the course is built in a very intuitive way. It touches many scopes at high level, some I liked more and some less, but that's how it usually work in the tech industry and it's a matter of taste I guess.

So I made a promise to myself: spend at least an hour every day learning new stuff. I know this is a very though challenge especially if you have a full time job and a family, but I decided that this is my last chance of doing this switch to SW development, and I'm going to do it.

There's another un-written rule that is highly recommended while doing this kind of challenge, and that's share your work.

Most of the people are sharing their work in platforms like Twitter, Linkedin or blogs, but I'm not so active in those social network, so I decided to upload it to a GitHub repository and on each PR add two of my co-workers as reviewers. The main point wasn't to get feedback on my code, but to get the feeling of sharing my work and getting feedback on it. It help me commit deeply to this process.

### So here are some takeaways:

__Self-discipline is key__

If you take this challenge, you will need self-motivation to work on the challenge every single day. Write reminders to yourself. Ask people to check on you, commit to it using social network or like any other creative way you feel comfortable with.

__You are still leaning even if you don't actually typing__

Google/Stack Overflow/Reddit/Discord (Generative AI wasn't born yet) and trying different solutions still counts as leaning. This is how you actually deepening your knowledge. You can code for a full hour, but sometimes handling a bug by digging online on why it happened or how to resolve it isn't less important.

__Create your own environment__

Work on the OS you feel comfortable with, use the editor you like, and use the tools you like. I know this sounds obvious, but I've seen people trying to learn on Windows while they are Linux users, or using VSCode while they are PyCharm users. This is a very personal thing, and you should feel comfortable with your environment especially when you are in the start of your learning journey.

__Take it Day by day__

100 days is a long journey. Try not to think about the 100 days as a whole, because that might be overwhelming. I for example, used not to look past the current section I was working on. I used to think about the next section only when I finished the current one. This helped me to focus on the current task and not to get overwhelmed by the amount of work I still have to do.

__Try to combine it with another positive routine__

In my case I printed and hanged on my desk a week schedule for the nights. Each day I had either learning or working out. This helped me balance the cognitive load.

__Don't be afraid to ask for help__

I used to ask many questions in the Python's Discord server and it provided many references and gave me lots of confidence that was missing.

__Document and share your work__

I already outlined it, but I can add that besides assigning reviewers, I also added a proper description in a README per each day which turned it to more meaningful for me.


It took me around 3 months to finish the entire course, which i'm very proud of and I got to a point where I felt comfortable with Python. But really quickly, I got to a conclusion that there's a large mile of being an actual developer and I had a feeling that i'm still not fully satisfied. I got to a conclusion that programming is a muscle that you will always have to continue to flex and grow. 

![100daysofcodecertificate](/images/100daysofcode-or.jpeg)

[My 100 Days of Code repository](https://github.com/Kazaz-Or/100DaysOfCode-Python)

There began my learning addiction.

I started consuming other online courses related to my actual work place. After a while I officially moved to the role of Automation Developer and from there everything is possible.
Every day at work or in my private time, i'm learning new things such as frameworks, libraries, tools and more.

![udemy-certs](/images/udemy-certs.jpeg)

You may have heard about the term tutorial hell. This is a real thing. Only way I could get out of it was to start building my own projects without tutorials and tried to integrate what I've learned in my work place (when possible of course).

Even today while i'm a full time Software Developer, I know that learning can't be paused and I have to keep learning new things every day. And so the learning journey continues.

If you are in this the middle of this challenge, or planning to start with it, feel free to reach out to me and I will be happy to help you with any questions you may have.


The 100 days of code is a true challenge. I've decided to take it on and document my journey.

My 100 Days of Code Journey


Having worked in the software Quality Assurance for many years both hands-on and in different management roles, I've learned a lot about the QA world, especially working on different discipline (from hardware-embedded, through web & cloud, to mobile and desktop applications).

During my career i've joined a small startup company as a sole QA engineer with working with many developers and managers. During time we grew and I had to assemble a team from scratch. Because we were a start up company with a huge lack amount of time (obviously), i've decided to build an aim for building a very specific team, and for this to be possible, i've started working on my own QA Manifest that will fit the company's needs (and to be honest - my own needs as well as a team lead).

For those who are coming from the QA world, some of these guidelines might seems extreme but I believe that if you want to build a strong team, you need to have a strong foundation and bold guidelines.

So, here it is, i'll try to share as much as I can:

## Goals and Values

- We know our Products the best

- We know our Stack (you'll be surprise to know how many engineers I have interviewed that didn't which technology they are working with, from programming language to their cloud provider).

- We are aware of everyone’s work in our team (This will be one of the key tools for the QA engineers in the team to become enablers & enforces).

- We aim for quality

- We are the enforcers of our squads & enablers of our developers.

- We optimize the Development & Testing process

- We balance between quality and cost efficiency 

- We ensure proper documentation and communication

- We aim for Automated testing 

- We aim for continuous improvement in all aspects

- We are the quality gate keepers of our company

This manifest is basically the foundation of the QA team, and it's the QA team lead/manager responsibility to make sure that the team is following it.

## Test Plan Process

Test plans are acting like an anchor for designing, execution, reporting, tracking and other activities related to the testing project. The below are queries related to test plan which should guide us on how to start building our test plans.

__What guides us while building our test plan :__

- Analyze the product (Feedback & questions for Product)

- What needs to be tested ? (Scope)

- How testing will be performed ? (Manual / Automated / Semi-Automated)

- What is required ? (Testing tools, Internal tools, Clients, Environments, etc.)

- Are there any Risks ? (Focusing on complicated areas of our product will help mitigate risks)

- Our bugs (old & new) are covered as part of the test plans? (Will help increase the coverage & create more deep dive tests)

- Taking our product / SW to its extreme edges (If it’s performance, edge cases, etc.)

- Try to focus on the user’s point of view (Is the UX sufficient in your eyes? who will use the product? What will be the main use cases? etc.)

__What should be our main References :__

References to various documents which are used in the company, such as :

- Requirements specification (Epics, stories, specs, etc.)

- Design document (from HLD, Flow Diagrams, technical specs, etc.)

- User guides

- Process/product documents

- Developers feedback

- Reference from previous versions and comparison (GA’s for example)

- Bugs

__How to Define Tests Objectives and what will help us do it?__

Test Objective is the overall goal and achievement of the test execution. In most cases the objective of the testing is finding as many software defects as possible; ensure that the software under test is bug free before release, enhance our confidence in the product and ensure our product is stable even in extreme cases.

To define the test objectives, it is recommended to do the following steps:

- List all the software features (functionality features, performance, GUI…) which may need to test.

- Make sure the expected result of each use case is compliant with high standards.

- Check that the external interface of the product such as UI is working as expected and & meet the customer need (UX). (You can see the example taken from a demo test app (Guru99 bank) below).

- Splitting the tests by their testing type (which at the release phase we can even decide which types we would like to test, such as Sanity, Happy path, Negative, Edge Cases, Error Cases, etc).

![guru99-demo](/images/guru99_demo.png)


| Advantages      | Disadvantages |
| ----------- | ----------- |
| Better and clearer test execution reports      | Not easy to divide to features in an efficient way       |
| Can provide us a better view on the “bad” areas in our product    |        |
| Can provide us coverage strength per feature   |      |
| Helps us differentiate between releases and coverage improvement   |        |


__Test Deliverables:__

- Test plans (to ensure coverage)

- Test execution reports 

- Bug Reports 

- Dashboards (Coverage, bugs, content, etc) 

- Test summary reports

- Contribute to release notes if required. 

- Handover of the release if required

- Summary email to wider distribution of relevant people

## Test Plan Reviews

Test Plan review is a critical part on our way to create a high standard test plan for our product/feature.

Test plan is not a document to cope with hands down. On the one side, it is difficult to create it and not to miss important information or carry it so that the client gets all the details. On the other side, you should have the experience to check a test plan to make sure it covers all the necessary details and that the testing provider gave the relevant information. That's why a review is fairly important.

__Types Of Reviews__

- Reviewing your own work – Self Checking

- Peer Review - Within the team (including developers)

- Supervisory - Being handled weekly in a QA forum (chapter, guild, etc.) with QA Engineers from other teams and the QA leads.


__When / Who / What / Why?__

_Who_ - 
Peer review should be according to the team effort and scope. In most cases it’ll include all QA engineers within the team, Team Lead and a relevant SW Engineer.
Is some cases the entire team take part is such review, but it might be according to the scope of the test plan - It’s yours to define what is the most effective way.

_When_ - Because it can be overhead, I would recommend doing such review in three cases - 

Reviewing test plan for a specific features (basically once a set of user stories are completed in terms of QA). I would recommend to do more deep dive into the tests (tests are kept as a simple flow, expected result is as defined, etc).

Reviewing at the end of each Sprint (by definition the review can be more high level to see we’re not missing anything from the scope or not testing too much out of the scope & and enhancing user scenarios).
Also, deciding what can be translated to Automated test.

Reviewing test plan for release - See that we’re testing to a point of striving to the maximum quality of the delivery which is in our scope. You should focus on features to be tested and high level review of the test cases.
Also, if there are cases which we decided not to include some tests / features in our test plan - Review this decision also.

_What_ - 
So what is our guidelines in terms of test plan review?

- Ensure that we satisfy all project requirements

- Ensure that we’re fully covering the functionality

- Ensure that the test are also exploring edge & error cases to see how our product reacts in extreme cases or smart ass users.

- Ease of understanding

_Why_ -

For exactly the same reason we test the software - 

- To uncover errors

- To check for completeness

- To make sure the standards and guidelines are as expected

- Approaches to consider for test design

- In order to ensure the product quality

- Tests cases are a great documentation to share our testing coverage (for both manual and automation tests)

## Testing Convention

__Generic Tests with Clear headlines and minimal steps (Breaking down steps into tests)__

Tests should be Generic as much as we can with a clear headline, so in most cases we won’t have to drill down to each tests to understands it’s scope and the reason of failures.

One of my golden rules, if a step can fail by itself, it means it should be covered as a test in most cases.

There are couple of emphasizes to make a generic test plan

- Each test should include it’s feature name at the start of the test name (this is mostly for reports / tracking purposes). 

- Each testing variable in the test can be split into different test which will make it easier to differentiate in case of failures.

- Less steps breakdowns - In most cases, if a step can potentially fail - It means its should be a test by itself instead of a step as part of a bigger tests. 
We do this to insure we know which TEST was failed and not which step. In addition, steps should be generic as much as we can (not fully detailed).


Let's see an example below from one of my test plans - 

![testplan](/images/testplan-matrix.jpeg)

Do we really need steps? Or is it clear by the headline itself?

On which test plan it’ll be clear enough to understand exactly what test failed?

On which one we have more chances to miss something?

Of course, this method won't fit in any feature/product, but again, from my experience, it's a good practice to follow because most of them will fit.

Let's take a look at some cons and pros of this method:

| Advantages      | Disadvantages |
| ----------- | ----------- |
| Easier to see the big picture      | Less convenient for new employees       |
| Enhancing the coverage per feature    | Requires strong documentation (which we’ll do regardless)         |
| Separation per a tests variable    | Less detailed test, in some cases will result in a longer learning curve        |
| More coverage    | Preliminary work on test plan will require additional time         |
| Less time consuming when analyzing failures    |       |
| Less “misses” on verification and regression tests    |        |
| Helps us cover edge cases     |     |
| Easier to maintain     |     |
| “Forces” engineers to fully understand the workflow and guidelines of the product     |     |

__Linked to Bugs (almost all bugs should be covered by tests)__

As best practice, almost every bug / ticket with QA verification can be translated into tests. In this case we know for sure we have coverage for a use case that was already defected and/or helps us not to miss any use case for future executions.

__Technical Details (like how to) should be linked to our knowledge sharing platform__

Lots of engineers like to fill this data in their steps, which eventually makes it much harder to maintain. In our case, we provided a link to the relevant Confluence page which includes all the technical details.
Regardless of tests plans, we must “own” the feature. Huge part of it is a strong documentation that we can modify in one place and share with other team members.

__Unlike Waterfall model, tests order (or hierarchy) isn’t a must. We should be able to execute tests as a stand alone without any pre-condition of other test cases or test suites__

At some point, we would like to test a specific test or a feature in a manual or automated way. Working without pre-conditions will help us execute tests in a more agile way.


__Test Structure and mandatory fields:__

Each test should include the following:

- Test Header should include clear scope of the test -

Feature - Test Type (for example; Sanity, Performance, Negative, etc) - Test Name (or Actions, depends) - Test Parameter (if required)

For example from my old test plan:

VOD Playout - Sanity - Play VOD File - HLS CLEAR

- Description - A short description of the purpose of the test. Link to a How-to Confluence page if required.

- Feature field - Each test should be related to a specific feature that we will be able to select from a list in Jira.

- Component field - Free text - We should add the Component/Microservice/Process that is relevant for the test to pass.

- Test Type - Manual / Automated.

- Fix/Target Version - The version that we added the test

- Linked Issue - Tests that are related to a specific bug that was translated to a test, should be linked to the bug, event if it’s an old bug that is already closed.


## QA Workflow & Methodology in Squads/Scrum Teams

General guidelines:

- QA Engineers should aim to have have their own “production-like” testing environment (if possible of course).

- As best practice, almost all Stories / Tasks / Bugs should be assigned to QA verification with clear testing instructions (if required) and regression testing recommended if needed (Anything that might have a functional effect on the product).

- QA Engineers should be on top of every change merged in their team, make sure you’re always aligned and on top of the current status.

- Make sure issues found are documented using Bugs in Jira, this will help track the status, and help us improve test plan.

- We have to be on top of the bug list in our domain.

- In terms of bugs, we have to use a clear description and try to find the balance between clear description for debugging purposes and for customer impact point of view mainly for people that are not part of the squad.

- We have to be clear about Reproducibility - For example: it the bug persists every time? Sometimes?

- We have to have a clear scenario and steps to reproduce.

- We have to set Severity / Priority according to business logic.

- When possible, we need to add as many evidence as we can, such as log, status code responses, screen shots, videos, etc.

- Control all bugs in the team, make sure you’re familiar with them, make sure from time to time they are relevant.

- QA Engineers should work closely with Automation engineers to make sure their coverage is being covered in an efficient way also in automated test runs.

- Infra Automation should be user-friendly so anyone in the squad can enhance out automation coverage in the team (QA, DEV).

There are plenty more guidelines, but these are the main ones that I believe are the most important. I had great success and with this method and I hope it will help you as well even if you agree with some of them.

In addition to that, one of the articles I liked to reference for new engineers in my team is this article - [Is There Any Start And Stop Boundary To The QA’s Role In Scrum?](https://www.softwaretestinghelp.com/qa-role-in-scrum/).

Thanks for reading, and I hope you enjoyed it!


As a former QA Engineer/Team Lead/Manager, I've learned a lot about the QA world. Here are my thoughts on what makes an excellent QA team

My QA Manifest As a QA Team Lead/Manager


I'll start with a personal perspective on this, and I know it might sound cheesy, but that's the how I look at it when looking back at my journey in the tech world.
Switching to a Linux distribution (my case Windows to Ubuntu) was a turning point in my career.

Switching to Linux was more than a mere change of operating systems for me. It was a transformational journey that broadened my technical horizons and significantly impacted my growth as a software developer. As a former Windows user and QA engineer, I was accustomed to a certain work process and mindset. However, taking the leap to Linux opened up an entirely new realm of possibilities.
Don't get me wrong, it wasn't easy at first, but eventually it was worth it.

Firstly, Linux introduced me to the power and efficiency of scripting. By utilizing fairly simple but yet effective bash script.
I could automate many of my testing processes, enhancing productivity and accuracy. The ability to manipulate tasks via a script was a game-changer for me, driving me to delve deeper into programming and the nuances of my system.

Moreover, Linux unlocked a new dimension of community engagement. This was not just about using a new operating system, but joining a global collective of like-minded enthusiasts and professionals. By participating in discussions on platforms like Discord and Reddit, and listening to informative podcasts, I gained a wealth of knowledge and insights. The ability to share, learn, and grow in such a diverse and dynamic environment is truly one of the most fulfilling aspects of being part of the Linux community.

This transition to Linux was not simply a switch in tools; it was a paradigm shift that changed how I approached my work. It provided a deeper understanding of computing principles, exposed me to a broad spectrum of technical paradigms, and connected me a lot more with the software world. And to be honest, I even believe that at some point it made me believe that I can code due to the learning curve and problem solving that I had to do.

__If you're still debating, I prepared a list of reasons why you should switch to Linux right now__:


### Open Source: The Heart of Innovation
Perhaps the most significant advantage of Linux lies in its open-source nature. The source code of Linux is freely available for modification and distribution. This grants engineers an incredible degree of freedom. Not only can you inspect the code to understand the system's inner workings, but you can also modify and optimize it to suit your specific needs. This level of customization leads to vast opportunities for learning and innovation, aligning perfectly with the creative and exploratory nature of a software engineer's work.

### It's The Backbone of the Internet
It's fairly simple, most of the world's servers are running on Linux. From GCP, Azure to Amazon, Linux is the backbone of the Internet. As a software engineer/QA engineer, you'll likely be working with these systems, so it's essential to be proficient in Linux to handle these systems effectively.
P.S, even if you're not working with cloud servers, Linux is still the most popular and widely used for Embedded, IoT and or any other system that you can think of.

### Security and Stability
Linux boasts an outstanding reputation for its security and stability. Unlike other popular OSs that are frequent targets for viruses and malware, Linux is far less susceptible to such threats due to its user privilege model and diversity. Stability is another strong suit for Linux; it can handle numerous processes running simultaneously without a hitch. This trait is crucial for software engineers who need their systems to perform consistently under heavy workloads.

### Superior Command Line Interface
Ah the CLI - Best thing ever. Once you get used it, you can't do anything without it!
For software engineers, the command line is a powerful tool. The Linux command line, also known as the terminal, is exceptionally powerful and flexible, allowing engineers to perform complex tasks with a few keystrokes. This efficiency can lead to improved productivity and a deeper understanding of the tasks at hand.
It also gives you an enormous amount of control over your system, and it's a great way to learn how your system works or even modify it to your needs.

![cli](https://images.unsplash.com/photo-1629654297299-c8506221ca97?ixlib=rb-4.0.3&ixid=M3wxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8fA%3D%3D&auto=format&fit=crop&w=650&q=30)

### Wide Variety of Distributions & Desktop Environments
The beauty of Linux lies not only in its wide variety of distributions but also in its diverse range of desktop environments.
From Ubuntu and Fedora to Debian and Arch Linux, each distribution offers unique features and caters to specific user preferences. Whether you value hard-core customization, stability, or ease of use, there's a Linux distro to match your needs.

In addition, you have the freedom to choose among numerous desktop environments such as GNOME, KDE Plasma, Xfce, and Cinnamon, to name just a few. Each offers a distinct user experience and different levels of customization. You might prefer the modern and streamlined look of GNOME, the feature-rich and customizable nature of KDE Plasma, or the lightweight and simplicity of Xfce.

![my-ubuntu](/images/myubuntu.jpeg)

### Thriving Community and Great Support
The Linux community is known for its vibrant and supportive culture. This global community is always ready to lend a helping hand to newcomers. For software engineers, this means that help is only a forum post away. Whether you're dealing with a software bug or looking for a new tool, the community can be an invaluable resource.
I can't tell you how many hours i've burned on Reddit and Discord, and it was totally worth it.

On a side note, I'm still hoping Reddit will get it together and realize they're product won't worth much without its core community [(if you don't know what i'm talking about)](http://redd.it/1476ioa)

### Tools Galore: All You Need and More
Linux offers a endless amount of tools like Docker & Git (and much more of course) that form the cornerstone of any developer. By mastering Linux, you're also learning to work and understand these powerful tools effectively.

### Efficient Installation of Applications and Tools
One of the key advantages that Linux provides is the ease with which software applications and tools can be installed. Linux distributions often come with package management systems, like `apt` for Debian-based systems such as Ubuntu, or `yum` and `dnf` for Fedora. These package managers handle the downloading, installation, and updating of software, drastically simplifying the process for users.
With a simple command, you can install your desired software without having to search for and download it from the web. 

Moreover, for software not available in the official repositories, third-party repositories and `snap` or `flatpak` application managers offer a wide range of software easily installable from the command line (or GUI). These methods ensure that almost any software you might need is just a command away.

### Cost-Efficiency
Last but not least, Linux is __free__. What else can I say? You get access to a robust, reliable, and high-performing operating system without spending a dime.


## In conclusion
Linux offers a multitude of benefits for software engineers. Its open-source nature fosters innovation and learning, its security and stability ensure reliable performance, and its free cost makes it accessible for all. With a wide variety of distributions and a supportive community, Linux offers an environment where software engineers can thrive.

If you haven't already, perhaps it's time to consider making the switch.


Linux opens possibilities, ignites curiosity & unlocks potential. For me, switching to a Linux distro resulted with a transition from QA a SW dev

Why You Should Switch or Embrace Linux Right Now


During one of our lunch breaks at work, we got into a discussion about the Y2K bug. I was surprised to find out that some of my younger colleagues had never heard of it. I decided to write this article to share this critical moment in software development history.

At the dawn of the 21st century, an enormous problem loomed in the tech world. This issue was famously called the Year 2000 problem, or the Y2K bug

This event served as a significant lesson for all software developers, prompting a fundamental shift in how we approach design and testing.

## The Y2K Problem:

The Y2K bug was "created" from a data storage decision made by programmers in the early days of computing. To save valuable memory space, dates were often represented only by the last two digits of the year. For example, 1990 was stored as '90'. This practice worked fine until we approached the year 2000. Under the two-digit system, the year '00' could be interpreted as 1900 rather than 2000, leading to potential software errors and system failures.

Now, let's try to simplify the issue with a small Python function:

```python
# Hypothetical function to calculate person's age in 2000
def calculate_age(birth_year):
  current_year = 2000
  # if birth year is '85', it interprets it as 1985
  # but if it is '00', it interprets it as 1900 instead of 2000
  age = current_year - (1900 + birth_year) 
  return age
```

`calculate_age` is a function that calculates someone's age in the year 2000, based on their birth year. It takes the `birth_year` as an input, which is last two digits of the birth year.

The function assumes that if the last two digits of the birth year are '85', for example, it interprets it as the year 1985. However, if the last two digits of the birth year are '00', it interprets it as the year 1900, instead of 2000 which it should be.

Because of the misinterpretation of the year 2000 as 1900, this function would return incorrect results for people born after 1999.

This is a simplified example to illustrate the type of problems that can arise from the Y2K bug, where systems incorrectly interpreted two-digit year values.

And this issue was causing a lot of panic around the world. This was mainly because of the severe implications - Banking systems could miscalculate, utilities might fail, even air traffic control systems could display incorrect data. Therefore, A massive, coordinated effort was needed to check, and if necessary, correct every piece of software that used a date function.

## The Fix:

Some of my younger colleagues were mistaken to think that this bug was a false alarm, because "nothing really happened". Well, that's incorrect.

Software engineers around the world were mobilized to mitigate the bug. Developers combed through millions of lines of code, looking for date-related functions to fix or upgrade. Many systems were switched from two-digit year formats to four-digit formats. In some cases, a temporary patch was applied - known as `windowing` - where a pivot year was established. If the system encountered a year below this pivot, it would interpret the date as belonging to the 21st century.

Let's take a look at the `windowing` solution in action:

```python
def calculate_age(birth_year):
  current_year = 2000
  pivot_year = 50  # pivot year is set as '50'

  if birth_year < pivot_year:
    birth_year += 2000
  else:
    birth_year += 1900

  age = current_year - birth_year
  return age
```

The function `calculate_age` calculates a person's age in the year 2000, based on their birth year, which is inputted as the last two digits of the actual birth year.
The key here is the `pivot_year`, which is set to '50'. This is used as a point of reference to decide how the two-digit `birth_year` should be interpreted.

If the `birth_year` is less than the `pivot_year` (i.e., between '00' and '49'), the function assumes that the person was born in the 2000s. Therefore, it adds 2000 to the `birth_year`. On the other hand, if the `birth_year` is equal to or greater than the `pivot_year` (i.e., between '50' and '99'), the function assumes that the person was born in the 1900s. Therefore, it adds 1900 to the `birth_year`.
Finally, the age is calculated by subtracting the interpreted `birth_year` from the `current_year` (2000 in this case).

This technique, known as windowing, was commonly used to resolve (or more accurate, "workaround") the Y2K bug in many systems.

What about the "real" solution?

The Y2K bug was essentially a data representation problem. The true solution to this was to change the way dates were stored, moving from a two-digit representation to a full four-digit representation of the year. This is often called the "four-digit year expansion".

Not sure it required, but if we started with code examples, let's demonstrate here as well (although it's the simplest one):

```python
# A function to calculate a person's age in 2000
def calculate_age(birth_year):
  current_year = 2000
  age = current_year - birth_year 
  return age
```

In this example, `birth_year` is expected to be a full four-digit year, such as 1980 or 2000. This change allows the program to understand the century in which the birth year occurred, which wasn't possible when years were represented with only two digits.

The switch to four-digit years was the most complete and future-proof solution to the Y2K bug, although it required significant time and resources to implement, as it often required updating a large amount of existing data and the programs that interacted with it. The need to complete this work before the arrival of the year 2000 led to a significant amount of pressure on technology organizations around the world.

Fortunately, thanks to the hard work of developers and a huge global effort, the world transitioned into the year 2000 with minimal issues. While there were a few minor glitches and system hiccups, the feared widespread failures and societal collapse didn't occur. The total cost of the Y2K fix is estimated to be over $300 billion, making it one of the most expensive software bugs in history.

The Y2K bug has left an indelible mark on the software development industry. It taught us the importance of forward-thinking in system design, the potential impact of seemingly minor decisions, and the need for deep testing procedures.


Explore the Y2K Bug that threatened global systems at the dawn of the 21st century and understand its impact on software development

Y2K Bug: A Look Back at a Critical Moment in Software Development History


How can we hide API keys, secrets and passwords in our code, especially if pushing our code to an external repository? How can we run an application locally using these env vars?

Preventing exposure of sensitive information in our code is essential and necessary - We don't want external parties to have unauthorized access to our databases, passwords, and services we use. Therefore, we want to secure this information as best as possible.

So how exactly do we do that?

The solution I used is quite simple - Wwe need to create a local file in the project root directory that will store and centralize all the API keys and sensitive information we want to secure. 

Additionally, we will ensure that this file is added to the gitignore file, of course, as it is intended to be stored only in the local environment.

This file called .env, which is short for environment variables. Its purpose is to manage and allow runtime access to variables containing sensitive information, without using hardcoding in the text for these values.

The file will contain only variables in a key-value pair format.

For example, in Node.js, to access the values ​​instead of writing in our code like this - const API_KEY = "123", we will write like this - const API_KEY = process.env.API_KEY (using the global Node object -> process).

Not only does the env file improve the modularity and maintenance of the code, but it also allows flexibility and separation between the values ​​of each key in different environments - for example, for the development stage, we will provide a separate env file with different passwords than in the testing or production stages.

In Node.js project I use the **dotenv** package to make life easier.

The dotenv package is a popular tool that simplifies the process of swapping various env file parameters in the system according to the work environment.

You can install it with npm (saved under dev dependencies)

```js
npm install dotenv --save-dev
```

And just for the sake of the example, here's how I run my unit tests using dotenv & Mocha.
From the package.json:
```js
"test": "mocha --inline-diffs tests/unit-tests/ -r dotenv/config"
```

In conclusion, you should definitely consider using env files to secure your  code, especially in personal projects and during the development stage. 


How can we hide API keys, secrets and passwords in our code, especially if pushing our code to an external repository? How can we run an application locally using these env vars?

All Blog Posts in the General Category