Blog of Sebastian Häni

What do Architects do?

Sun, 07 Apr 2024 18:00:00 GMT

This blog post reflects my thoughts on what is architecture, what does an architect do and what they should focus on. I copied in some quotes which I found enlightening. The sources can be found in the references. It is also extending the thoughts from this older post: Decentralized Architecture.

What is Architecture?

Is architecture the taking of Important Decisions?
→ Important is what an expert developer declared as important.
Therefore, importance is actually not a useful criteria for architecture.
Is architecture the Decisions to be made early?
→ Decisions to be made early can be eliminated.
Therefore, it is not decisions to be made early.
Is architecture the Decisions hard to reverse?
→ Yes.
Therefore, try to make decisions easy to reverse; reduce the number of hard to reverse decisions.

Therefore, get rid of architecture itself.

Getting rid of Software Architecture

The following three quotes are from Who Needs Architects — Martin Fowler.

It’s commonly believed that if you are building an enterprise application, you must get the database schema right early on because it’s hard to change the database schema—particularly once you have gone live. On one of our projects, the database administrator, Pramod Sadalage, devised a system that let us change the database schema (and migrate the data) easily.

→ We have to find abstractions to make changes easy.

Complexity is what makes software hard to change. That, and duplication.

→ We have to simplify.

Software is not limited by physics, like buildings are. It is limited by imagination, by design, by organization. In short, it is limited by properties of people, not by properties of the world.
“We have met the enemy, and he is us.”

→ Hard to reverse decisions can be converted into easy reversible ones.

What do Architects do?

Improving the development team’s ability gives an architect much greater leverage than being the sole decision maker and thus running the risk of being an architectural bottleneck.

→ An architect’s value is inversely proportional to the number of decisions they make. → An architect best acts like a guide (as in mountaineering guide) and not as decision maker.

Implications

Principles

These principles directly relate to the discussed topic:

Architect’s Activities

Ensuring that conversations that are needed to be happening are happening
Ensuring decisions are of high quality
Guiding when needed
Reducing irreversible decisions

How can an architect do that?

Conversations happen & decisions are of high quality (see Decentralized Architecture):
- A thinking and recording tool: Architectural Decision Records (ADRs)
- A time and place for conversations: Architecture Advice Forum
- A light to illuminate a unified goal: Engineering Principles
- A tech landscape and current climate sensing tool: Tech Radar
Guiding when needed:
- Show presence
- Connect people
- Be available
- Be useful
- Act timely on guidance requests
- Be cautious about decisions made in the dark
- Reflect and and distill principles with teams
Reducing irreversible decisions
- Promote principles
- Be cautious of irreversible decisions and guide towards reversible ones
- Spend time thinking about abstractions to create systems with “reversability” built-in.

Approach

Let’s use a city analogy:

We can design the blueprint for a single building in a city.
Or we can plan the layout of the entire city, including roads, buildings, and utilities.

If buildings are small and simple and can be extended easily, i.e. are built with evolution in mind, we maximize reversible decisions whenever we want to change the building or even demolish it.

So, how can we foster buildings being built this way that satisfies these properties? We can guide each building builder to design their building to satisfy these properties, but that takes time and leads to bottlenecks.

What if we instead build the best infrastructure for buildings? The provided infrastructure acts as a virtual restraint for the buildings. If there is a water pipe provided, the building builder is unlikely to pump water out of the ground and build their own water treatment plant, they’ll just use what’s already provided. This requires that the infrastructure is flexible, extensible and reliable.

Fostering an environment where systems are built with evolution in mind and where reversible decisions are maximized is crucial for adaptability and scalability.

To achieve this:

Every component must be simple. Software and system components should be modular and loosely coupled, allowing for easy replacement or modification without disrupting the entire system.
Off-the-shelve solutions and standards are so good, they do not need enforcement to be adopted. Frameworks, protocols, and interfaces promote interoperability and reduce the need for custom solutions.
Allowing focus on what matters for the business. Automation tools and processes streamline development, testing, and deployment, enabling rapid iteration and evolution of software systems.
Have shared understanding. Clear documentation and effective communication channels ensure that design decisions are well-documented and understood, facilitating future modifications and enhancements.

Minimal Viable Architecture

In his talk Minimal Viable Architecture, Randy Shoup explains that there are roughly three phases when building a system:

The city analogy is generic, and one has to understand in which phase the system is in.

The city analogy is more fitting for the scaling phase, where the focus is on building the best infrastructure for buildings. In the starting phase, the focus is on building the best building. In the optimizing phase, the focus is on consolidating the buildings.

In other words:

in the starting phase, focus on software architecture is more important.
In the scaling phase, focus on system architecture is more important.
In the optimizing phase, focus on software architecture is more important again.

During the scaling phase, a greater focus on system architecture as in platforms, infrastructure, automation, tooling, conventions, nudging, processes, and facilitation all with the goal to make decisions more reversible is likely to provide greater positive impact in a world where time is limited.

Striking the balance is still necessary though, it’s just the that there is more weight on system architecture.

To find the right balance follow this in this order:

Align with Business Goals
Manage Risks
Consider Reliability and Quality Aspects
Use Feedback-driven Iteration

That’s a strategy. The implementation of the strategy will have it’s pitfalls and learnings.

References

Architectural Significance

Wed, 02 Aug 2023 18:00:00 GMT

The following describes what can be an architecturally significant decision interpreted from Architectural Significance Test written by Olaf Zimmermann.

Short Form:

One of:

High Business Value or Risk
Important Stakeholder

… and at least one of:

QoS Deviation
External Dependencies
Cross-cutting Impact
First-of-a-Kind (FOAK)
Historical Troubles

Long Form

If the decision you want to take meets these criteria, writing an ADR is most appropriate.

One of these criteria apply:

The requirement is directly associated with high business value (benefit vs. cost) or business risk.
The requirement is a concern of a particularly important stakeholder such as the project sponsor or an external compliance auditor.

… and at least one of these criteria also apply:

The requirement includes runtime Quality-of-Service (QoS) characteristics (such as performance needs) that deviate from those already satisfied by the evolving architecture substantially.
The requirement causes new or deals with one or more existing external dependencies that might have unpredictable, unreliable and/or uncontrollable behavior.
The requirement has a cross-cutting nature and therefore affects multiple parts of the system and their interactions; it may even have system-wide impact short term and/or long term (examples: security, monitoring).
The requirement has a First-of-a-Kind (FOAK) character: for instance, this team has never built a component or subsystem that satisfies this particular requirement.
The requirement has been troublesome and caused critical situations, budget overruns or client dissatisfaction on a previous project in a similar context.

Decentralized Architecture

Mon, 24 Jul 2023 18:00:00 GMT

Central Architecture as executed by a single architect or a group of architects has three main problems:

Bottlenecks — Having a central authority which designs architectures or even if it only has to approve it, slows down delivery.
Ivory Tower — The central authority does not have to live with the consequences. Neither does a birds eye view suffice to solve problems adequately, because the devil is usually in the details.
Conway’s Law — The organization structure defines the architecture. Designing without being inside the organization is not possible. The result will be different.

This post discusses how to decentralize architecture to address these problems.

The content is a summary of the excellent blog post by Andrew Harmel-Law Scaling the Practice of Architecture, Conversationally.

Decision-making via the “Advice Process”

This is the most fundamental change.

The Rule: anyone can make an architecturally significant decision [1].

The Qualifier: before making the decision, the decision-taker must consult two groups: The first is everyone who will be meaningfully affected by the decision. The second is people with expertise in the area the decision is being taken.

[1] An architecturally significant decision is one that is hard to change later, has a significant impact on the system, and is not obvious. See Architectural Significance for more.

We are not looking for consensus here, but we are looking for a broad range of inputs and voices. Decision-takers are encouraged to specifically seek out those who will disagree with them.

When deployed, decisions are better, faster, more accountable, and most importantly decisions which are understood and owned by those who implement them, precisely because the decision-taker is the one with the need as well as the one who is accountable.

The traditional architect now plays the role of a Guide, drawing on their experience in the field.

[In order for an architecture to be successful] it is very much about ensuring that conversations that are needed to be happening are happening - not always initiating them, nor always helping to focus or navigate them, but ensuring they do happen […] and guiding when needed

— Ruth Malan

Also check out this excellent talk: Architecture without Architects - Erik Dörnenburg

The Four Supporting Elements

The Advice Process requires four elements to be implemented successfully:

a thinking and recording tool
→ Architectural Decision Records
a time and place for conversations
→ Architecture Advisory Forum
a light to illuminate and guide towards a unified direction
→ Team-sourced Architecture Principles
a means to sense the current technical landscape and climate
→ Tech Radar

The Core and Supporting Elements of Decentralized Architecture

Architectural Decision Records (ADRs)

A thinking and recording tool

ADRs are lightweight documents per decision taken that was architecturally significant.

Elements of an ADR when used in an advice process context:

Name	Description
Title	Which includes a unique identifier, and the decision itself (e.g. “ADR001 - Use AKS for Kubernetes Pods”)
Status	Typically “Draft”, “Proposed”, “Adopted”, “Superseded” and “Retired”
Decision	The decision that has been taken in a few sentences (frequently bold or italicized so it stands out)
Context	The forces and current contextual circumstances which have necessitated this decision
Options Considered	Each option considered, described briefly, with pros and cons. (Typically the option proposed / adopted comes first in this list)
Consequences	The ramifications of this decision, both positive and negative
Advice	This reflects the raw outputs from following the Advice Process. It is here that all advice given is recorded. This ought to include the name of the advice giver, and the date the advice was given. This can frequently take the forms of comments, and if these are provided directly by the advice-giver, then recording the meta-data is automatic.

Architecture Advisory Forum

A time and place for conversations

The Architecture Advisory Forum is a regular and recurring place and time for conversations.

Goals: High quality architecture conversations take place in a timely fashion. Equally important is the breadth and diversity of views shared, and the same goes for the contributors. The goal is to “do” architecture and share and learn lessons.

Attendees: Ideal attendees are delegates from each team as well as key representatives from the Advice Process. However, the invite should remain completely open to encourage transparency and openness.

Team-sourced Architectural Principles

A light to illuminate and guide towards a unified direction

What are good principles?

Clear Direction — You should be able to make a decision based on the principles, almost like an algorithm.
Timeless — They should not change often.
Universal — They should apply to a variety of situations.

The set of principles should cover the key needs, and should not be too many that the teams cannot remember them all — and crucially the teams need to feel a sense of ownership.

Important:

Principles are not Practices (such as TDD, Trunk Based, Pair Programming).
Principles should not be too general (such as Write Good Code)

Example: John Lewis IT Software Engineering Principles

Tech Radar

A means to sense the current technical landscape and climate

The Tech Radar is a tool to help teams understand the current technical landscape and climate.

To build your radar there are a few tools:

Or just use Miro or any other collaborative diagramming tools.

In a context with a wide variety of technologies, it can be useful to have a radar per technology area.

Conclusion

Decentralized Architecture is a powerful way to scale architecture in an organization. It is not a silver bullet, but it is a way to ensure that the right conversations are happening, and that the right decisions are being made.

Maven Dependency Tree

Tue, 21 Mar 2023 18:00:00 GMT

Ever wanted to understand the dependency graph in a Maven project with sub-modules?

Add the following to the parent pom.xml:

<project>
  <build>
    <plugins>
      <plugin>
        <groupId>com.github.ferstl</groupId>
        <artifactId>depgraph-maven-plugin</artifactId>
        <version>4.0.3</version>
      </plugin>
    </plugins>
  </build>
</project>

Run mvn depgraph:aggregate -DcreateImage=true [-Dincludes=groupId:artifactId]
The graph can be found at target/dependency-graph.png

Documentation: https://github.com/ferstl/depgraph-maven-plugin

Example

(Click to enlarge)

Introduction to Distributed Tracing

Sun, 10 Jul 2022 18:00:00 GMT

Distributed Tracing

Distributed Tracing enables us to follow invocations across services and monitor them. We can observe a single service using logs and metrics. But what if we have a chain of services that fulfill a single purpose for an end user? For example if we have a user that clicks on an order button in a web frontend, we call the frontend’s backend BFF (I personally favour the Backend-for-Frontend pattern) which internally calls an inventory microservice to see if the product is available, and also calls a billing microservice to get the user’s bill they will need to approve. If the billing service returns an error, the BFF returns an error to the user. However the inventory microservice returned a successful response. If we only have logs and metrics, we do not know what we just described, we simply know there were some errors, but their correlation is not given. With tracing we know that an error in a service very likely lead to another error in the consumer service if they are in the same recorded trace.

Source: https://medium.com/nikeengineering/hit-the-ground-running-with-distributed-tracing-core-concepts-ff5ad47c7058

A trace has an ID and consists of one or more spans. A span has an ID, timestamp, duration, a local and a remote endpoint as well as a name. For REST calls we traditionally use the HTTP method and path as the span name, e.g. GET /info. So, how can we collect spans that form a trace from different services running in different containers? The service that starts a trace is generating a trace ID and with every request it creates in the trace context, it not only creates a span, but it also sends the generated trace ID as an HTTP header. Zipkin specified the span propagation headers first and termed them B3.

   Client Tracer                                                  Server Tracer
┌───────────────────────┐                                       ┌───────────────────────┐
│                       │                                       │                       │
│   TraceContext        │          Http Request Headers         │   TraceContext        │
│ ┌───────────────────┐ │         ┌───────────────────┐         │ ┌───────────────────┐ │
│ │ TraceId           │ │         │ X-B3-TraceId      │         │ │ TraceId           │ │
│ │                   │ │         │                   │         │ │                   │ │
│ │ ParentSpanId      │ │ Inject  │ X-B3-ParentSpanId │ Extract │ │ ParentSpanId      │ │
│ │                   ├─┼────────>│                   ├─────────┼>│                   │ │
│ │ SpanId            │ │         │ X-B3-SpanId       │         │ │ SpanId            │ │
│ │                   │ │         │                   │         │ │                   │ │
│ │ Sampling decision │ │         │ X-B3-Sampled      │         │ │ Sampling decision │ │
│ └───────────────────┘ │         └───────────────────┘         │ └───────────────────┘ │
│                       │                                       │                       │
└───────────────────────┘                                       └───────────────────────┘

See B3-Propagation to learn more about B3 headers. Other vendors followed suit with their propagation format, which meant there were suddenly multiple out there. Nowadays, it’s best practice to follow the Trace Context W3C Recommendation. If unsure, or during migrations, you can include different propagation headers in your requests, as they don’t bite each other.

The service answering this request picks up the headers and does not generate a new trace ID, but instead uses the provided trace ID. If this service calls yet another service, it simply passes on the trace ID it received. This is handled by instrumentation libraries and a developer usually does not have to care about this, except configuring the tracer library. OpenTelemetry provides state of the art implementation libraries that we can use. They provide compatibility to OpenTelemetry Collector, Prometheus, Jaeger and many others and support different transports (TCP, gRPC, UDP) and different deployment models (in-application, agent, …).

So, each service now knows to which trace its spans belong to, but how do we collect this information? This is where the Collector comes in. We could tell our services to send the spans to an OpenTelemetry Collector or to a Jaeger collector or any other collector that we’d like. Given the common trace ID, the tracing backend can then correlate the spans together and display them in a single timeline. Sending spans in batches to a collector can impact your service’s performance, so there are options to also do this using a sidecar / agent approach, which involves running another process on the same host next to the service container or even a process running in the same container using process daemons.

Tracing data is complementary to logs and metrics and does not replace them.

Using tracing data, we can also visualize system architectures based on actual data in realtime. Especially with microservices and with chaos engineering, your system architecture diagrams will quickly be outdated. We can analyze the local and remote endpoints of each span and determine the direction of communication between two services. Aggregated over all spans we get a dependency map. If we index this data we can even query for the map during a time frame and even visualize throughput and error rates on edges between services.

Tools

Instrumentation:

OpenTelemetry Instrumentation *recommended
Jaeger - deprecated
Zipkin - deprecated
AWS X-Ray

Collecting:

OpenTelemetry Collector *recommended
Jaeger - preferred over Zipkin
Zipkin - first one to be widely adopted
AWS X-Ray - if you’re running services on AWS

Analysis / Post-Processing:

Jaeger - supports wide range of storage backends, decent UI *recommended
Zipkin - first one to be widely adopted, scaling issues, UI bugs
AWS X-Ray - if you’re running services on AWS, automatic integration into lots of AWS services

Others: Tanzu Observability, Dynatrace, Azure Application Insights, Azure Monitor, Graphite, Datadog, Google Cloud Monitoring, Lightstep, Logz.io, Sentry, Splunk APM

System Architecture

My personally recommended architecture looks like this:

The service, of which there can be many, is deployed using an agent (if available for the language and framework). This is then sending the tracing data (and metrics) to the OpenTelemetry collector, which has a configured pipeline to receive spans and forward them to the Jaeger collector, which stores it in a storage backend such as an Elasticsearch database. The Jaeger Query service allows an engineer to query the traces and visualize them.

OpenTelemetry Introduction

Sun, 10 Jul 2022 09:30:00 GMT

What is OpenTelemetry

OpenTelemetry has been formed as a merge by the teams of OpenTracing and OpenCensus while providing full compatibility to existing technologies in the field.

OpenTracing has created a set of vendor-agnostic APIs to collect tracing data in applications running primarily as Cloud Native. This means that a developer who wants to collect tracing data in their application does not have to implement a vendor’s API in their code, but instead uses an open API that is then implemented by the respective vendors. Similar to Slf4J as a logging API that allows Log4J, Logback etc. to be used as a logging backend. OpenCensus is a project by Google to collect observability data through standard formats that are also vendor agnostic. Thus, OpenTelemetry is a project that creates vendor agnostic APIs for instrumenting applications and at the same time collecting that data from multiple sources and only then passing it to a vendor’s tool.

So, if you are using Prometheus and Jaeger today, you might want to switch over to using OpenTelemetry APIs and the OpenTelemetry collector as an intermediary, and only then send the tracing and metrics data to your observability tools. With this abstraction, you can use multiple observability tools in place at the same time, route data to different places, sample data, etc. All of this is independent of the specifics of vendors. However, you are then dependent on OpenTelemetry.

The observability signals for OpenTelemetry are:

Logs: Traditional logs printed to console, file, or in case of a cloud native application, to a log-collector which indexes them and provides a search interface. If logs are written in the context of a trace, the trace and logs can be linked and made discoverable together.
Traces: A trace consisting of spans, which are basic function calls, follows the path of invocations no matter in which service they are invoked.
Metrics: Timeseries based data. For example traffic count, error count, response times, etc

Additionally, OpenTelemetry specifies Baggage which are basically tags for events.

As of July, 2022, only Traces and Metrics have been fully specified. They both have well tested instrumentation libraries in the most common languages and frameworks. The OpenTelemetry collector is also able to export these to common observability tools such as Prometheus, OpenCensus, Zipkin and Jaeger as well as Kafka. Logs are not fully specified nor implemented, but it’s being worked on right now. Check out the OpenTelemetry Status Page to see an up-to-date status.

The need for standards

In 2010 Google has published a paper about Dapper, a Large-Scale Distributed Systems Tracing Infrastructure. Some called it the secret weapon Google has to understand their architectures and debug complex request traces.

Dapper later evolved into the open sourced project Zipkin. AWS jumped onto the distributed tracing bandwagon and introduced AWS X-Ray. Uber liked distributed tracing as well, but didn’t like some decisions taken in Zipkin, so they created Jaeger.

https://xkcd.com/927

Each of these tools introduced span propagation methods to be able to follow a trace as it makes it’s way through an architecture consisting of many services:

Zipkin B3

x-b3-sampled: 1
x-b3-spanid:  9090044efa29991f
x-b3-traceid: de5a14e8f69781b31299211599430ed2

AWS X-Ray

x-amzn-trace-id: Root=1-de5a14e8-f69781b31299211599430ed2;Parent=9090044efa29991f;Sampled=1

Jaeger

uber-trace-id: de5a14e8f69781b31299211599430ed2:9090044efa29991f:0:01

Zipkin later introduced a single header standard as well:

Zipkin B3 single-header

b3: …

Finally, some people were annoyed by the variety of propagation formats, so they created a W3C recommendation:

Trace Context W3C Recommendation

traceparent: 00-de5a14e8f69781b31299211599430ed2-9090044efa29991f-01

However, the software APIs to instrument your code and to collect those signals was still wild west. So, OpenTelemetry was formed to come to the rescue.

Instrumentation

Before OpenTelemetry, you could choose to use a library to instrument your code to gather tracing and metrics data and collect it with your vendor’s tool. For example, for tracing you could use the Zipkin libraries, Jaeger libraries, AWS X-Ray libraries or possibly others.

Now, these often provided compatibility to other tools, however you still needed to depend on a single vendor. And to be fair, libraries such as Spring Cloud Sleuth abstracted the vendor library for you, but you still needed to include it in your application.

Nowadays, Zipkin and Jaeger have deprecated their instrumentation libraries and recommend the usage of OpenTelemetry instrumentation.

Collection

Before OpenTelemetry, you had to send your metrics, traces and logs to separate vendor specific tools.

For tracing data collection vendors introduced these formats:

Zipkin Format
Protobuf
Thrift

And transport it over these:

HTTP(S)
gRPC
UDP

While the transports probably all have a justified use case, the formats should be unified. So, OpenTelemetry introduced the OpenTelemetry Protocol OTLP. It not only allows to send tracing data, but logs, metrics and tracing.

Once they were collected by a vendor’s tool, it was hard to get them out of there to use it for other interesting purposes. In a large organization, it might be useful to send traces to a team specific tool, but also collect all traces produced by all teams to generate so called RED metrics.

R - Requests: Traffic, Throughput, Rate
E – Errors: Error Counts, Error Rate, Failed Calls
D – Durations: Latency, Elapsed Time

With the OpenTelemetry Collector in-between, this becomes a breeze, as you can define pipelines in the collector which sends your data to different channels downstream. If you ever want to evaluate a new monitoring tool, you simply forward your data there, but continue using your existing monitoring tools until you switch over. Or maybe the tools are complementary and you decide to use multiple tools generating insights based on the same source data.

Upcoming

This post was a first introduction into OpenTelemetry. I am planning to release a series of posts around this topic.

Introduction to OpenTelemetry (this article)
Introduction to Distributed Tracing
OpenTelemetry Collector Components (upcoming)
OpenTelemetry Instrumentation (upcoming)
Instrumenting React.js applications (upcoming)
Instrumenting Angular applications (upcoming)
Full Stack OpenTelemetry Example (upcoming)

Setting up Apache Spark and Zeppelin

Sun, 16 May 2021 18:00:00 GMT

This article describes how to setup Spark and Zeppelin either on your own machine or on a server or cloud. Spark and Zeppelin are big software products with a wide variety of plugins, interpreters, etc. Finding the compatible versions, Dockerfiles, configs, etc. for a simple setup can be daunting. This here might help you to deploy and play around with Spark on your local machine or create a server setup as a ground to do some demos or proof of concepts. In this setup we will use the local file system and not a distributed one. When you use Spark in a production grade setup, you want to scale your filesystem and workers. But, our goal here is to create a playground environment.

This blog post refers to this accompanying GitHub repository:

sebastianhaeni/spark-zeppelin-docker

What is Spark and Zeppelin

Apache Spark is an engine to process your data in a highly parallel and fault tolerant way. You can submit programs as Spark jobs which will be run on Spark cluster consisting of at least one worker node connecting to a distributed storage. The programs can be written most famously in Python (PySpark) or Scala using the MapReduce paradigm. Alternatively, you can use Spark SQL to submit SQL queries and retrieve the query result. However, the user interface of the Spark master and worker nodes merely display running jobs and logs. There is no user interface to submit jobs. This calls for a Notebook solution.

Apache Zeppelin is a notebook web application like JupyterLab focusing on notebook collaboration. Both tools cover a similar feature set. There are many opinions out there on which tool is better. Please make your own opinion about it. This article just covers how to install it. Zeppelin fits quite nicely into the Spark world, but you can achieve the same results using JupyterLab.

Image Source: zeppelin.apache.org

Setup

This diagram shows all the Docker containers we will setup:

Instructions

Note: You need to configure Docker to have at least 4 GB of memory.

Clone this repo git clone https://github.com/sebastianhaeni/spark-zeppelin-docker.git
Build the base image docker build -f Dockerfile_base -t registry.local:5000/spark-zeppelin-demo/spark_base:latest .
Build the remaining images docker compose build
Append the following to your /etc/hosts (Linux & MacOS) / C:\Windows\System32\drivers\etc\hosts (Windows) file
```
127.0.0.1 zeppelin.local
127.0.0.1 spark-ui-proxy.local
```
Run with docker compose up
Open http://zeppelin.local/ in your browser to access Zeppelin
Open http://spark-ui-proxy.local/ in your browser to access Spark

And there you go. An example notebook is included as well to get you going.

Reverse Proxy

The reverse proxy allows us to proxy to an internal container by server name. So, we don’t have to map ports from the internal Docker network and we don’t have to remember names.

To make this possible, we also need a custom Python web server that transforms the responses from Spark-Master and Spark-Worker. They do not support reverse proxies out of the box (no support for X-Forwarded-For etc.), thus we rewrite the <a> links to the correct URLs.

Spark and Zeppelin

Base Image

The Spark and Zeppelin images both depend on a base image, that we have to build beforehand.

This image contains:

JDK
Scala
Anaconda & Python
Apache Spark
Apache Zeppelin

The reason why we create a single image with both Spark and Zeppelin, is that Spark needs some JARs from Zeppelin (namely the spark interpreter jar) and Zeppelin needs some Spark JARs to connect to it. Also, Spark needs Anaconda (Python) to run PySpark. Zeppelin has a pure Python interpreter that also needs Anaconda (to be able to achieve something meaningful).

If you need to add another Python package to the image, you can do so either with conda install or pip install at the end of Dockerfile_base. Please be sure to rebuild the images after doing so.

Apache Spark

The current configuration will create the following 3 containers:

spark-master: Spark Master Node where workers connect
spark-worker-1: A worker node, connecting to the master. Could be scaled.
- Please note that the amount of CPU cores and memory to be reserved by the worker are defined in Dockerfile_worker and can be optionally overwritten by environment variables.
spark-ui-proxy: To access the nodes’ web UI, we need to transform the content with correct URLs (they don’t properly support reverse proxies)

Please note that the Spark nodes have a volume mapping to ./data . This is where you can place files to read in your Zeppelin notebook. An example file is already provided.

%pyspark
df = spark.read.json("/srv/data/example/people.json")

Apache Zeppelin

Zeppelin will be connected to the Spark Master (Spark Interpreter) once you run the first Spark cell in a notebook.

Compatibility Issues

Please note that Spark, Zeppelin, Python, JDK, Scala have to be compatible to each other. In the current setup, these have been tested and matched to each other. For example, this would not work:

Upgrade from JDK 1.8 to e.g. 11. You will run into some class not found issues when running Zeppelin.
Upgrade from Python 3.7 to Python 3.8. You will notice weird errors when running a PySpark script.

So if there’s a need to upgrade, please reserve enough time to test the compatibility between those systems well enough.

A good approach can be a test notebook in Zeppelin containing blocks with Scala, Python, PySpark and SQL interpreters. Run them all successfully, and you basically proved that the system works.

Orgazining optimized code reviews in your team

Mon, 28 Sep 2020 20:00:00 GMT

In the teams I worked so far, we always opted for full reviews of each engineer’s code. But how do you organize these reviews in a fashion so that they:

are efficient - do not take more time than necessary
improve code correctness and quality
improve code readability
find bugs
allow for knowledge spread

We discussed in our team that we would like to create a consensus on what Change Reviews mean for us and how we apply them. The following is the documentation that I created for our team.

This has been influenced by the Google’s Engineering Practices documentation.

And further, how do you contribute to a code base in an aligned way? I have added some contribution guidelines as well. The contribution guidelines are inspired by Angular’s guidelines.

And how do you finally map your status into a project management tool like JIRA?

Additionally, we also are holding regular know how exchange sessions or discussions related to tech and engineering to ensure we have a common vision and understanding of recent or upcoming topics.

Change Reviews
Contribution Guidelines
How we use JIRA
Tech Discussions

Change Reviews

Review Planning

How to define the reviewer?

In general, there should be two reviewers. The main reviewer does a thorough review, while the second reviewer is there to catch any big mishaps and to ensure knowledge sharing throughout the team. It might happen that there is only one reviewer in case of smaller tasks.

Code Review is always required

Each change has to be reviewed, even minor ones. The reason for reviews of big changes should be clear. Small changes should be reviewed to ensure nothing has been accidentally introduced that could break things. E.g. there’s a faulty merge in the change that the author was not aware of. These small reviews may be as short as 30 seconds, but they should still occur.

Review Execution

Review Handover

We regard the handover from the author to the main reviewer as one of the most important aspects of a successful review. The author has to make clear what they expect from the reviewer.

In general, there can be an “early look” review to guide the author in the right direction. For this, the build has to be successful to make the reviewer’s effort worthwhile. The integration tests are ok to be still running or failing if they take longer. In the end there always has to be a final review. For the final review, all builds and tests have to be successful and all merge conflicts resolved.

The author can optionally annotate their changes to explain complex factual relations. This will enable the reviewer to be faster when trying to understand the changes.

During the handover, it should be made clear if the reviewer is allowed to push changes onto the branch. This can make sense if the reviewer wants to commit small changes, such as:

cosmetic changes
fixing comments
fixing typos
very small code fixes
rebase / merge onto the current develop branch if there are no conflicts or they are trivial

To fasten up the review process, the author or the reviewer can also request a pair review. During the pair review, they can push changes directly within the review. After this, no further ping pong is necessary. A change can then be merged as long as the builds and tests are successful.

Review Feedback

In case of an async review, the reviewer creates comments in the PR. If they want the author to change something, they create a task below the comment indicating action is needed. The change cannot be merged if there are incomplete tasks.

If the reviewer has not set any status on the PR yet, this means the review is in progress, and the author should refrain from modifying the branch. Once the reviewer sets the status to “Needs work”, the ball is back at the author to complete the tasks in the comments. Once the reviewer is happy, they set the status to “Approve”, and the author can go ahead and merge the changes.

Review Contents

The most important content of the review should be discussed in the handover. Besides, the reviewer must:

understand the user story
understand each and every line of code in the change
understand the commit history

The reviewer can test the code by clicking through the staged environment. They can also check it out and tinker around in the code and tests. If this makes sense or not can be different from case to case.

What the reviewer is not doing, is to clarify story requirements with any other than the author such as the product owner. They can or even should however discover if the story description diverges from the implementation such that the business requirements are not met.

Review Completion

Merge

After the main reviewer gave their final “Approve”, the author can merge the changes.

Follow-Ups

There always may be follow-ups after the change. This is something that should generally be avoided. A follow up can be a TODO/FIXME in the code, a bug, a missing feature, technical debt, ignored tests and so on.

If it happens, that it cannot be avoided, and the reviewers accept the follow-ups, these rules have to be observed:

Each follow up must exist as a JIRA issue
Commits introducing the follow up have to mention the JIRA issue
TODO and FIXME comments must contain the JIRA issue (e.g. // TODO fix label not being displayed (AAA-xxxx))

PO Review

The PO review for larger user stories is planned at latest during the daily stand up meeting. The author has the lead.

UX Review

In a UX intensive task, it is generally a good idea to consolidate the UX designer and get some feedback. This should happen before any code review occurs.

Contribution guidelines

Commit message template

Template for commits related to a JIRA issue:

AAA-<number> (<type>) <subject>

Explain why this change is being made

If there is no issue, use the following template:

(<type>) <subject>

Explain why this change is being made

Types:


feat	new feature
fix	bug fix
refactor	refactoring production code
style	formatting, missing semi colons etc; no code change
docs	changes to documentation
test	adding or refactoring tests; no production code change
build	updating pipelines, *.gradle, package.json etc; no production code change
deps	updating maven or npm dependencies
i18n	updating translation files

Example:

AAA-0000 (feat) implement X validation logic

This introduces 2 new components:

* X-tile
* X-edit-form

The service `XService` has been extended to support
updating and removing X entities via the backend.

Remember to

not capitalize the subject line
Use the imperative mood in the subject line
Do not end the subject line with a period
Separate subject from body with a blank line
Use the body to explain what and why vs. how
Use multiple lines with ”-” for bullet points in body
Squash commits that make it hard to read the history.
Don’t squash commits if they contain different types as defined above.

Branching Model

We are using the Git Flow branching model.

See https://nvie.com/posts/a-successful-git-branching-model/ for a very nice blog post explaining it very well.

Feature Branches

Feature branches should be named with the prefix feature/ as per recommendation of Git Flow. Additionally we include the JIRA Issue number and a short description of the feature.

E.g.

feature/AAA-9999_Group-Easteregg-Tab

This provides these benefits:

Bitbucket will automatically link the branch with the JIRA issue and vice versa.
We can easily jump between requirements in JIRA and the branch.
With the description, we know what’s on a feature branch even without looking up the issue description.

Other prefixes:

release/ reserved for branches for a version (also for hotfixes, albeit we deviate from the guide)
bugfix/ reserved for fixes only (refrain from doing any refactoring here)
hotfix/ for hotfixes that will be merged into the hotfix-release branch
doc/ reserved for docs-only changes

Docs Branches

If you only update documentation you can prefix your branch with doc/.

E.g.

doc/knowhow-transfer-update

So, only use it if you are certain you only committed docs changes.

How we use JIRA

Before a sprint

We split stories into multiple stories and create an epic for it if the story has an estimate of over 40 hours before or during PI planning.
We create sub-tasks for each story which has an estimate of less than a work day.
Sub-task descriptions can be filled with additional hints or implementation detail. They should however not contain redundant information from the parent story.
We do not create extra review sub-tasks unless they are really needed.

During a sprint

We continuously move cards in the board so it is up to date all the time.
We log hours worked on a task when we move a card to another column and reduce the remaining estimate in accordance.
An individual contributor can create new sub-tasks in their story as long as the total sum of remaining estimates stays the same. The reshuffling has to be discussed with the main reviewer.
If the estimate for a task changes (up or down), this has to be discussed with the PO or Scrum Master as soon as it is detected.
When a developer is done with implementing a sub-task, they move the task to the review column.
If they reached a checkpoint which can be reviewed, they inform the reviewer personally and create a PR and assign the reviewer(s) to it.
The reviewer moves the task to the To Do column if the review is not OK or completes the task(s) together with the author (someone merges the PR) and moves the task to the Done column.
The reviewer also logs work hours on the task they reviewed.

PO Review

A story can have a PO review or not. It is up to the PO and the team to decide if it makes sense or not.
If a story should include a PO review, we create a sub-task in the story with an empty estimate.
This sub-task is then moved along the columns on the board when it is ongoing or complete.

Tech Discussions

To ensure we have a common vision and understanding in technical topics regarding our services that we build, we regularly hold tech discussion meetings. Everybody is invited to add their topics to a topic list. Every developer can then vote on topics. After the event is over, we clear all votes and it starts over.

The Tech Discussions help us to improve:

common understanding of technical aspects
know how sharing of new concepts
discussion of technical directions we should take
presentation of results of spike undertakings

Action items are tracked as JIRA issues. The documentation of discussed topics always has to be up-to-date and ensures the discussed items are not forgotten over time or decisions can be looked up again.

Conclusion

Maybe you discovered some practice which could help you in your team?

It certainly helped me a lot having this written down. Even we discussed everything in the entire team and everbody agreed to it, it occurred sometimes that the discussed items were forgotten, and having this document helped us to quickly resolve questions that we have already answered a year ago.

Seminar Assessment on Oneself

Tue, 07 Jul 2020 17:00:00 GMT

In my first blog post related to the AI seminar that I took, I laid out my learning goals. Now the seminar is coming to an end, and I will try to assess my own performance and review if the goals have been met.

Goal Verification

Here are the goals once again. So, let’s go through it.

1. I want to know in detail how the process of writing scientific papers is like and hear from an expert that has experience in this.

Thanks to our lecturer who is a journal editor himself, I learned a lot of the process that scientific papers go through, and I’m grateful for those insights. And by practicing certain aspects of the process myself, such as reviewing a paper, I got to understand the details of it as well.

💯 goal fulfilled

2. I want to try criticizing a paper and get feedback.

In my last blog post I reviewed the paper I was reading. During this activity I had to research the actual structure of a review, and I think I captured it well. And I also learned about how others approach this activity.

During our session we talked about the reviews, and I got a bit convoluted with what I could actually improve. My note taking was sub-par at that point which is why I’m now unsure how I faired.

🔳 goal fulfillment unsure

3. I want to refine my writing skills.

Writing blog posts for a broader audience definitely helped to refine my writing skills. Each time I dedicated a lot of thinking time into the structure of the post. My process was as follows:

Lay out the structure of the post. The question I asked myself was, how should the reading journey be like? How do we start and how do we end? What does the audience expect? I also did some research how similar posts are structured and if I can improve on that.
Fill in the created sections with bullet points of what I want to write about. This helped me to understand at an early stage if the flow of the post will make sense in the end. So here I also reviewed the structure again and made some readjustments.
On a different day, the next step would be to fill in the sections with prose text, images and references. While writing, I often did research and looked for interesting graphics or references.
My last step is proof-reading and ensuring the post forms a whole and sentences or paragraphs are not dangling.
Publish

I learned that reading the sentence that I wrote again on a different day usually gives me ideas on how to improve it to support the reading flow of the reader. I’m still having trouble to stay away from passive sentences though.

Looking at the writing style of the paper I read was also helpful, as I think it’s style was quite good, and I identified the parts that were not so good.

💯 goal fulfilled

4. I want to get to know the method explained in my chosen paper in detail.

I think now I understand the contents of the paper “Forecasting at Scale”, because I not only did a skimming read, a thorough read and reading of surrounding literature, I also summarized it in a blog post and held a short presentation.

One could look at this process of summarizing like an autoencoder. A summary should be shorter than what you are trying to summarize, otherwise it’s not a summary. So somehow you have to make it shorter, and you do this by compressing information down. The person reading the summary should still be able to grasp the content of the original paper with some slight imprecision when filling in the gaps in their own mind. That’s exactly what an autoencoder architecture would accomplish as well. So the summarizing process is figuring out what parts have to be encoded into the summary for a possible decoding to be successful.

Image Source: mc.ai

In turn I discovered the key points of the paper and internalized it for myself.

💯 goal fulfilled

5. I want to establish a regular blogging habit beyond this seminar.

I can’t judge this goal just yet. I have written one post that is not related to the seminar as yet and I have more topics in my queue. I sure hope I will find the motivation to keep this going. I clearly see the benefit of doing so.

🔳 goal fulfillment not possible yet

Through this blog and colloquial sessions with my peers and lecturer I was able to share my learning process. This blog post itself is the culmination of it.

💯 goal fulfilled

Comparing myself to others

I was not the only one participating in this seminar, and we were specifically asked to compare our performance to others.

Here I will split my comparison into the different parts that we participated in:

Blog Posts: I tried to make my posts as enjoyable to read as possible with a reasonable time amount spent writing them. As I plan to continue using this blog as my personal blog for the foreseeable future, I always had in mind writing for a broader audience that is not exclusively limited to participants of the seminar. I was always impressed by what the others created and tried to learn from their good efforts and tried to improve upon. Unfortunately I almost never published my posts very early, which is why my colleagues had little to no time to read it before our sessions. So I would give myself a negative rating for the timeliness, but an above-average rating for the content itself compared to others.
Session Participation: I came reasonably well prepared into the sessions, having read the emails and slides from last time, the additional resources and the posts of the others. I enjoyed the sessions as they were focused and required one to be “present” with their mind. I think all participants did well.
Presentations: The long-ish presentation is still ahead of me. The few things I presented were ok-ish. At one time I was confused as what to present, because we were asked to prepare to present the findings of our writing style analysis and then in the session we were asked to present the content (without having thought about how to present it). I also think all participants did well in this regard.

Learnings Summary

So, I learned to:

read / understand a scientific paper
obtain more information / do literature research
put research in context (with respect to current state of the art in the field)
form my own scientific opinion / judgement
present scientific results in different ways (blog, orally, presentation, written document)
engage in a scientific discussion, express my view, formulate criticism

I can highly recommend this seminar to any other fellow students that might be reading this blog, thinking about enrolling. The university internal name of the course is “EVA Artificial Intelligence Seminar”.

A Review of Forecasting at Scale

Tue, 02 Jun 2020 22:00:00 GMT

This is my final review of the paper Sean J. Taylor & Benjamin Letham (2018) “Forecasting at Scale”, The American Statistician.

I’m going to follow a common peer review template that is in place at some conferences such as IJCAI while also explaining the score category further down this article, so you and future me know what questions should be asked oneself when giving a review.

A peer review always has to be in a context. This can for example be a certain conference or journal. So, I’m going to pretend I’m reviewing this for an AI conference similar to IJCAI (which might not make sense to ever submit this paper there, but I’m getting there 😄).

My Scores

Relevance: 5 / 10 ⭐
Significance: 5 / 10 ⭐
Originality: 1 / 10 ⭐
Technical Quality: 3 / 10 ⭐
Clarity and quality of writing: 8 / 10 ⭐
Scholarship, i.e. scientific context: 7 / 10 ⭐
Overall Score: 4 / 10 ⭐
Confidence on my assessment: 5 / 10 ⭐

My Comments About The Paper:

Summary

The paper “Forecasting at Scale” is a well laid out paper giving an insight into methods and techniques used to process business time series at scale, do forecasts with the analyst in the loop and automatically select models. The trend model incorporates dynamic growth rates and change points. The seasonality model is based on Fourier series. The authors propose to use a method called Simulated Historical Forecasts (SHF) to automatically evaluate forecasts. The main findings are a set of techniques to apply in a large corporation with analysts that might have their hands full of work to optimize their workflow to produce accurate forecasts with only little intervention.

The authors assembled their techniques from several previous work, and I think one that gives a great overview is “25 Years of Time Series Forecasting” which is also referenced by the paper. I regard the significance and novelty as rather low and think the authors produced a good confirmatory paper showcasing how these techniques work at scale.

The authors explained their chosen methods well, they make sense and offer what is promised. A reader gains a good feel to build a forecasting system that can work at scale.

What I’m missing in the paper are verification results of the chosen methods and how they would fare against other or even similar methods. Sometimes the authors also don’t explain their choices.

I greatly appreciate the accompanying Python and R forecasting library Prophet that has all mentioned methods implemented and is super fun to use. The paper is a good introduction for users of the library.

Major Issues

As mentioned in the summary, comprehensible verification of the chosen trend and seasonality methods is largely missing.
This paper is confirmatory of previous work and thus belongs into a journal rather than to a conference.

Minor Issues

Table 1 is superfluous. The description of holidays and their format is enough.
The style of figures should be consistent throughout the paper.
The remark that the authors are storing SHF in Hive or MySQL can be helpful but feels out of place. Either it should just be more of a focus to describe implementation details or leave them away completely.

Confidential Notes to Editor

Since this is not a real review and there’s no editor I will not write any notes here, but still include it so it’s clear a review usually includes this section.

One could mention that paper could be suspect to plagiarism or comments that the authors maybe didn’t know enough about previous work. If you are making a borderline decision you can also add some comments to make the score distinguishable to the editor. And, if you have confidential information that you would not like the authors to know, you can add it. This can for example be collected data that only your or your organization have and so on.

If you have a conflict of interest you should not even start the review and make it clear beforehand.

Review Template

The score categories I used above are explained here. The explanations are taken from the slide deck we received at my university ZHAW.

Relevance

Is the paper fully within the scope of the conference?
Will the questions and results of the paper be of interest to researchers in the field?

Significance

Is this a significant advance in the state of the art?
Is this a paper that people are likely to read and cite?
Does the paper address an important problem?
Does it open new research directions?
Is it a paper that is likely to have a lasting impact?

Originality

Reviewers should recognise and reward papers that propose genuinely new ideas.
As a reviewer you should try to assess whether the ideas are truly new.
Novel combinations, adaptations or extensions of existing ideas are also valuable.

Technical Quality

Are the results technically sound?
Are there obvious flaws in the conceptual approach?
Are claims well-supported by theoretical analysis or experimental results?
Are the experiments well thought out and convincing?
Will it be possible for other researchers to replicate these results?
Is the evaluation appropriate?
Did the authors clearly assess both the strengths and weaknesses of their approach?

Clarity and quality of writing

Is the paper clearly written?
Is there a good use of examples and figures?
Is it well organized?
Are there problems with style and grammar?
Are there issues with typos, formatting, references, etc.?
It may be better to advise the authors to revise a paper and submit to a later conference, than to accept and publish a poorly-written version. However, if the paper is likely to be accepted, please make suggestions to improve the clarity of the paper, and provide details of typos.

Scholarship, i.e. scientific context

Does the paper situate the work with respect to the state of the art?
Are relevant papers cited, discussed, and compared to the presented work?

Overall Score

(10) This is best-paper material
(9) An excellent paper, a very strong accept
(8) A very good paper, a strong accept
(7) A good paper, accept.
(6) A good paper overall. I vote for acceptance, although would not be upset if it were rejected because of the low acceptance rate
(5) Decent paper, but may be below the IJCAI threshold. I tend to vote for rejecting it, although would not be upset if it were accepted
(4) I vote for rejecting it, but could be persuaded otherwise–3-A weak paper, just not good enough.
(3) A clear rejection. I vote and argue for rejection. Clearly below the standards of the conference
(2) A very strong rejection. I will actively fight for rejection.

Confidence on your assessment

(10) My own current research is on the topic of the paper
(9) I have undertaken research on the topic of the paper
(8) I am an expert in this area
(7) I have up-to-date knowledge in the area
(6) I don’t have complete knowledge of the area, but can assess the value of the work
(5) I have a general understanding of the area
(4) My assessment is an informed guess
(3) My knowledge in the area is limited
(2) My knowledge in the area is very limited
(1) My assessment can be wrong

Comments to Authors (free text):

General points:

Make sure you substantiate all the scores for the different criteria above.
Give informative content.
Be constructive.
Write your comments in the same way you would like to receive comments on your own papers.

The comment section can be laid out like this, taken from wiley.com - How to perform a peer review:

Summary

Give positive feedback first. Authors are more likely to read your review if you do so. But don’t overdo it if you will be recommending rejection

Briefly summarize what the paper is about and what the findings are

Try to put the findings of the paper into the context of the existing literature and current knowledge

Indicate the significance of the work and if it is novel or mainly confirmatory

Indicate the work’s strengths, its quality and completeness

State any major flaws or weaknesses and note any special considerations. For example, if previously held theories are being overlooked

Major Issues

Are there any major flaws? State what they are and what the severity of their impact is on the paper

Has similar work already been published without the authors acknowledging this?

Are the authors presenting findings that challenge current thinking? Is the evidence they present strong enough to prove their case? Have they cited all the relevant work that would contradict their thinking and addressed it appropriately?

If major revisions are required, try to indicate clearly what they are

Are there any major presentational problems? Are figures & tables, language and manuscript structure all clear enough for you to accurately assess the work?

Are there any ethical issues? If you are unsure it may be better to disclose these in the confidential comments section

Minor Issues

Are there places where meaning is ambiguous? How can this be corrected?

Are the correct references cited? If not, which should be cited instead/also? Are citations excessive, limited, or biased?

Are there any factual, numerical or unit errors? If so, what are they?

Are all tables and figures appropriate, sufficient, and correctly labelled? If not, say which are not

How to Forecast at Scale

Tue, 12 May 2020 23:30:00 GMT

This is my third blog post about the paper Sean J. Taylor & Benjamin Letham (2018) “Forecasting at Scale”, The American Statistician.

I’m going to focus on its contents and try to place it in the field of forecasting and business time series.

Business Time Series

Most large corporations process a lot of interesting data in the form of Business Time Series. These can be customer related such as purchases, interactions through consultations, support, incident reports, usage of the service and so on. They can also be about internal processes that cannot be attributed to a single customer, such as infrastructure providing services to multiple customers.

These time series can provide insights on how the company performed in the past but also provide the foundation to create forecasts. The paper “Forecasting at Scale” provides the reader with practices proofed in the real world because they have been applied at Facebook. If the proposed methods work outside of Facebook and are applicable to other organizations has to be found out.

Preliminary reading

I’m going to list a few topics that you should be familiar with in case you decide to read the paper.

Linear Regression, Advanced Regression Methods, Exponential Smoothing Methods, Bayesian Statistics, Fourier Series, STL

If you want to reproduce the results of the paper, I recommend you are familiar with:

Stan, R, Shiny

Assembly of Ideas

The paper has the main idea to assemble several methods into a scalable forecasting system. So what are the ideas that the authors have assembled to build this system of scale?

A forecasting model
Automatic evaluation of forecasts

The Prophet Forecasting Model

As we know from the STL paper (Seasonal and Trend Decomposition using Loess), one can try to decompose a time series into trend and seasonality components using Loess regression.

Trend

The mathematical expression to compute the nonlinear, saturating growth trend in its most basic form is

with C the carrying capacity, k the growth rate, and m an offset parameter.

This gives us a formula that supports a static carrying capacity and static growth rate. In the real world capacity and growth rate are often not static and need to be dynamic. We can replace the carrying capacity with an analyst-specified dynamic capacity function based on time. The growth rate can be replaced with a vector that contains the manually specified change points with the respective growth rate of that time interval. This gives us the new formula

Sometimes we only need a piecewise linear model. This formula can be reused when simplified. It will give us a linear trend with change points by removing the carrying capacity, and a constant growth replaced by rate adjustments.

The uncertainty of this model is computed by using it generatively, after having inferred the model parameters, to forecast the future, while assuming the rate of change points is going to have a similar average cadence. We can repeat the generation while sampling the model parameters from a Laplace distribution and create an uncertainty region.

Seasonality

Since seasonality in the real world is mostly a composition and does not have a fixed window in which it repeats, it can be flexibly modeled using Fourier series. If we have different components for different periodic effects, such as yearly, holiday or weekday components, we can create an additive model based on that.

A standard Fourier series can approximate arbitrary smooth seasonal effects with:

Fitting

The complete model can be fitted in Stan:

The analyst creating a fit has to tweak the model parameters to reach the right compromise between loss and overfitting. An analyst has settings to adjust capacities (C), change points (X), holidays and seasonality (A, gamma) and smoothing parameters (tau, sigma). Often times this requires a broad experience in the business field to come up with good model parameters. This can for example be because the analyst has to have a feeling for the impact of an upcoming holiday in terms of trend or seasonality change.

The analyst is supported by visualization that show the effects of the latest changes to their model:

Automatic Evaluation of Forecasts

The analyst can optimize a model given time. If there are too many forecasts that have to be produced that can be handled by humans, the automatic evaluation of forecasts comes into play.

The metric used to compare models is the mean absolute percentage error (MAPE) over a number of forecasts produced with different prediction horizons.

Time series data points are inherently dependent on previous points. This is making it hard to use traditional methods such as cross-validation. The method proposed is Simulated Historical Forecasts (SHFs), that produce K forecasts at various cutoff points in the history. The generally used amount of points is a forecast every H/2 periods. This allows the model to produce an error term using MAPE which can be compared to baseline models.

Time series data usually encompasses a lot of data points and running SHFs with the maximum amount of periods would not be feasible as the model needs to be retrained each time. With SHFs, the simulation count is significantly reduced and the computation can be run independent on separate machines.

Paper Context

The paper “Forecasting at Scale”, picks up common knowledge that the field of business time series has been using for quite some time but mix it with other techniques to solve new challenges while making it scalable and allowing the analyst to be in the loop.

In my opinion the value the paper creates in the field is not the paper, but the companioning library Prophet which makes all those techniques readily usable for a broader audience. The paper is worth a read if you’re going to use the library. If you’re not planning that have a go with it, I can still recommend it as there might be some smallish novel ideas introduced that you might find interesting. If you are already well versed in the time series field, I would say that this paper is not going to contain much new for you.

Paper Follow ups

The library Prophet is under active development for over a year now. The open source community contributes to future improvements to the code. The basic methods mentioned in the paper implemented haven’t been changed though. So I’m guessing that they are a good fit for the real world.

Automatically keep your dependencies up to date in closed source projects for free

Sun, 26 Apr 2020 20:00:00 GMT

We tackle the question of why it’s important to keep your dependencies up to date, compare a few proprietary solutions and then talk about how to do it with NPM, Gradle and Jenkins yourself.

Why it is important to keep your dependencies up to date

There are several reasons why you should keep the dependencies you use in your project up to date:

Keep the burden of migrating small in case of incompatibilities: If you do not regularly upgrade your dependencies you will eventually have to and face a big list of breaking changes which do not take minutes to fix but hours, days or weeks. By keeping your dependency versions close to the latest versions, you can upgrade in small steps, and the change you need to do to your code base to make it compatible, is small each time. So, I prefer small incremental migrations that are easy to be covered by a maintenance task in your backlog rather than big migrations that take up the whole Scrum sprint. Because big migrations are harder to sell to your business owners than unnoticeable migrations. If you wait too long with upgrades you might even stick with what you got. This is the first step to doom the code base, as it’s eventually going to be rewritten by fresh developers not accepting to work on a legacy code base.

Profit from the latest features and performance improvements: Keeping up to date comes with the benefit that you can always rely on using the latest features the dependency provides and generate customer value in a modern, possibly easier or more reliable and faster way. It keeps the developers working on the project happy to work with the newest versions and allows them to brag to colleagues from the field that they are up to date.

Ease of updating in case vulnerabilities get found: Imagine a vulnerability gets found in one of the dependencies you rely on to run your application. And the good thing is, the library owners have fixed the dependencies in the latest version. But then you find yourself to be back 3 major versions which did not receive a patch to fix this vulnerability. So, what are your options? You can upgrade the library and spend all the migration effort right there and then and put everything else on hold, or just accept the risk. Neither are easy to sell to your business owner. If you are using the latest versions of your dependencies, the upgrades cost you almost zero time.

How recent should you keep dependencies

Generally, I would say that keeping your dependencies at the absolute latest version, including major, minor and patch is the preferred way. Don’t use betas, release candidates or other non stable versions. Although using the absolute latest doesn’t always make sense depending on several factors. If you’re using libraries or platforms that have a huge user base, it’s likely the authors have different release channels for different users.

One release channel often is the Long-Term Support (LTS) channel offered. If the authors provide such a channel, it generally is a good idea to stick to the LTS versions of the dependency you are using until the authors switch LTS to a newer version. Examples for platforms or libraries offering LTS is: Java, Node.js, Angular, whereas for example Angular has LTS for the two most recent major versions to allow you to upgrade within half a year to stay within LTS versions:

Angular's support policy and schedule

Once you upgrade your versions you should always verify your system works as before, doesn’t have negative impacts and maybe even measure if it profits from performance improvements mentioned in changelogs of the upgraded dependencies. You should also have a look at what has been deprecated and will eventually be removed in a future version, which gives you the benefit that you are aware of bigger migrations and lets you plan them ahead of time.

To verify compatibility a good test suite, ideally including unit and integration tests, comes into play. If you don’t have any or sufficient tests, you don’t have enough confidence to update dependencies regularly. That’s obviously a problem, and you should look into increasing the coverage of your tests.

In any case, automated tests probably won’t find 100% of problems, so I would say it’s a good idea to upgrade major versions just after your latest major rollout to production. If you use a branching strategy like Git Flow that would mean you update on the develop branch. This allows for all testers and developers working on the new version to experience the new major version of your dependencies during development. They can find incompatibilities before you ship to customers.

Dependency Update Automation Bots

There are several products offered on the Internet to automate dependency updates for you. Here’s a list of a few of them:

They work great, and work in the way, that they have a scheduled job to update your dependencies, create a pull request and let you decide if you want the update or not. The problem with them is, that they are usually free for Open Source projects on GitHub or other hosting platforms, but as soon as you want to automate updates for your closed source project you have to pay, and sometimes these are considerable amounts. Sometimes they don’t even support your existing CI systems.

Luckily, there is no need to pay for these services, you can just build it yourself with all the control you want. The following sections describe how I built dependency bots for our team.

How to automate updates with your automation system such as Jenkins

The basic principle is that you have an automatically scheduled job run by your automation system, such as Jenkins, that looks at your dependencies, does a diff with the package registry or repository and updates your files specifying the new versions, then creates a commit on a new Git branch and pushes that to the Git repository. With the new branch it will create a new pull request, at which point your CI system picks up the pull request and runs your tests against it to see if they still pass. If you’re happy with the updates, you merge the pull request and there you go. You can now update your dependencies with a single click while having confidence they didn’t break anything.

So, the question is, do you want exactly one job updating all of your outdated dependencies at once, or do you want to introduce update groups. In my project, I decided to introduce 4 update groups:

NPM packages: scheduled nightly against develop branch
Major, Minor, Patch updates for Open Source Gradle dependencies: scheduled nightly against develop branch
Minor, Patch updates for Open Source Gradle dependencies: manually triggered if needed against master branch
Internal Gradle dependencies: scheduled every 6 hours against develop branch
- In our project, we use software libraries from other teams of the company in our code that are updated multiple times per day, and we want to integrate with these changes just a few hours after they have been released.

The extreme choice would be to create update groups for every single dependency that you have. But I didn’t do that, because:

Firstly, in NPM, several dependencies, such as Angular’s, must be updated in accordance to be compatible
Secondly, to spare our build infrastructure and not overload it with tests to execute, I decided to update dependencies that come from public sources every night and group those together in one group.

The drawback is, that if two dependencies are updated within one day and one of them breaks your build, you cannot merge the pull request for the one dependency that works.

A script automating this could look like this for NPM:

#!/bin/bash

# create new branch and checkout
git checkout -b feature/npm-update

# do an install so we know what versions we have present
npm install ; EXITCODE=$?
if [ $EXITCODE -eq 1 ]; then
    echo "npm install failed!"
    exit 1
fi

# list outdated packages
npm outdated

# do the update
npm install ; EXITCODE=$?
if [ $EXITCODE -eq 1 ]; then
    echo "npm install failed!"
    exit 1
fi

# run npm install again to ensure package-lock.json is stable
npm install ; EXITCODE=$?
if [ $EXITCODE -eq 1 ]; then
    echo "npm install failed!"
    exit 1
fi

# stop if there are no changes
if git status | grep -q "working tree clean" ; then
    echo "No updates available! That's good :)"
else
    git add -A
    git commit -m "(deps) update npm dependencies"
fi

Then add these two steps as post actions:

Either add a git push command, or in case of Jenkins run the Git Publisher plugin to publish the branch.
Add code that automatically creates a Pull Request in your Git repository.

How to automatically update NPM dependencies

If you want to automatically update your NPM dependencies, you already have the tools from NPM itself, no need for other plugins. You can run npm update to update the versions in your package.json file according to the version selector you have defined.

If you don’t know about NPM’s version selectors syntax like ^, ~ and others, I encourage you to try out the playground to learn about it. If you want a default save prefix for new packages installed, you can create a .npmrc file in your project directory with the line save-prefix="~" added to it. This changes the default ^, that allows updates to new minor versions to ~, which only allows updates to new patch versions. That’s not something I would recommend doing, as that would restrict your updates more. If you use save-prefix=">=", every time npm update is run, you get the newest production ready version (don’t worry, you won’t get betas, release-candidates and so on).

Another thing to note is that NPM only updates direct dependencies that you specified in your package.json file. Transitive dependencies (dependencies of your dependencies) will not be updated. You can include transitive dependencies with npm update --depth 9999. I would actually not recommend doing this on a nightly basis, because the amount of updates you will receive is going to be immense. So, I would resort to only let your direct dependencies specify their minimal versions of transitive dependencies and not update them yourself. The exception might be when one of the transitive dependencies have vulnerabilities, and you want to fix them in your application before the direct dependency is updated.

By the way, the recommended depth 9999 by the NPM documentation failed in my project because NPM ran out of memory. So, you might want to scale it back to a safe depth like 10.

So, all you need is:

npm outdated to print outdated versions
npm update to automatically update them

How to automatically update Gradle dependencies

If you’re using Gradle to manage your dependencies for Java, Kotlin or Groovy projects (or others?), then there’s no built-in way to update your dependencies, but there are great plugins to do so.

I’m actually using two plugins; one depends on the other.

The plugin gradle-versions-plugin is able to compare your versions to the versions available in the specified artifact repositories. It adds the task dependencyUpdates to your configuration which outputs the versions that can be updated in various formats.

The second plugin https://github.com/patrikerdes/gradle-use-latest-versions-plugin is then able to take the output of gradle-versions-plugin and find these versions in your build.gradle file(s) and replace it with the new version.

Additionally, you can configure exactly what updates you want, and what you don’t want through configuration in the Gradle files or through command line arguments.

In our build.gradle file I added the following snippet to keep non stable versions and allow them to update to newer non-stable versions but keep dependencies that are using stable versions today remain on stable versions.

Also, I added in an optional property onlyMinorPatchUpdates for minor updates. I run the update job with this property against branches that contain code already shipped to production, since I don’t want to introduce new major versions there.

tasks {
    dependencyUpdates {
        // consider BOM version definitions and do not complain about transitive constrained out-of-date versions
        checkConstraints = true

        def isNonStable = { String version ->
            def stableKeyword = ["RELEASE", "FINAL", "GA"].any { word -> version.toUpperCase().contains(word) }
            def regex = /^[0-9,.v-]+(-r)?$/
            return !stableKeyword && !(version ==~ regex)
        }

        def getMajorVersion = { String version ->
            return version.substring(0, version.indexOf('.'))
        }

        def isMajorUpdate = { String versionA, String versionB ->
            return getMajorVersion(versionA) != getMajorVersion(versionB)
        }

        rejectVersionIf { component ->
            if (project.properties.get("onlyMinorPatchUpdates") == "true"
                && isMajorUpdate(component.candidate.version, component.currentVersion)) {
                return true
            }
            return isNonStable(component.candidate.version) && !isNonStable(component.currentVersion)
        }
    }
}

I hope this article was useful to you. If you have any remarks, questions or recommendations, feel free to leave a comment.

Dissecting a paper's writing style

Thu, 23 Apr 2020 20:00:00 GMT

In this post I want to dissect a paper’s style and try to criticize it irregardless of it’s content. The paper is the one mentioned in the previous post:

Sean J. Taylor & Benjamin Letham (2018) “Forecasting at Scale”, The American Statistician.

Standard Structure

Let’s go over the standard structure of a paper and describe what to look for, and then giving the verdict of how the paper did.

0. Title

A good title for a paper should honestly reflect the content, give the reader hints what topics it will touch, and reveals the category of the paper (argumentative, analytical, definition, compare and contrast, cause and effect, report. interpretive, etc.).

Sometimes people working for big names put catchy titles on their papers such as “Attention Is All You Need” or “One Model To Learn Them All”. This yields that if that paper really is groundbreaking, people will remember it for the catchy name, and it’s easy to talk about it in colloquials. For the authors is a kind of branding for them as they can afford to add a title like this. A researcher with their feet on the ground probably has to resort to more bland titles that are humble and serve their purpose in the journal or conference.

Verdict: The title of the paper “Forecasting at Scale” is pretty short and does not summarize the techniques or methods described in the paper and rather focuses on the fact that the paper is practical experiences made since the word “Scale” is mentioned. So in my opinion it’s a bit catchy but that’s maybe in the nature of the authors since they work for Facebook and can afford to have a title like that. It’s clear the content will not be about biomedical topics or physics. It could also be about weather forecasting if it weren’t published in a journal named “The American Statistician”.

1. Abstract

The abstract is the first thing a paper, and the part most often read. It should stand for itself and not require the reader to read any other part of the paper to understand it.

The structure of an abstract is:

Problem: what is it and why it is interesting
Method: what and how is it accomplished
Result: what is the new achievement
Importance: what follows after the result

The abstract is for people that want to read the paper. It is not to be confused by the conclusion which is for people that have read the paper.

Verdict: The abstract gives a nice introduction to the field of forecasting time series in the first sentence. It makes me feel guided into the topic. The second sentence starts explaining the problem as it should mention according to our list above. From then on the abstract still follows the structure but tends to say “we will describe in this paper” and not actually giving the reader the result right there in the abstract. It’s nicely rounded off by explaining where these methods are now productively in place.

The introduction explains the problem in depth and why it’s interesting and important. It also explains why naive approaches usually fail and why other people have not solved it before and thus makes the problem hard. It should also introduce the proposed approach.

This section should also include related work with a brief overview of the current state of the art with explanations what is new with respect to previous work. It should be come clear what’s the improvement.

Verdict: The introduction is in itself structured logically, and the reader can easily follow. It explains what they mean by “scale”.

In my opinion the paper’s introduction and the main body diverge a little as the introduction explains scaling problems, and the main body describes solutions to forecasting problems for the good first part.

They don’t talk about why these problems have been solved by others though. They also don’t mention related work. I could imagine the reason being the other few companies that could encounter scales like Facebook don’t publish their results as white papers. Though I could also imagine the authors didn’t care much about related research work, since they were building a practical implementation that matches their company environment.

3. Main Body

This part should talk about formulation, implementation, results, experiments, etc.

The main body is a longer piece of content which should be grouped into sections. Ideally the text doesn’t introduce more than one idea per sentence or paragraph. A topic should be in its own section.

General good practice:

use short words
use short sentences with simple structure
keep paragraphs short
avoid buzzwords, clichés and slang
avoid excess, in length or style
omit unnecessary material
be specific, not vague or abstract
break these rules if there is a good reason to do so

Verdict: The main body is well-structured, sections are not too long, paragraphs have a reasonable length and contains a good amount of visualizations. Details as to why they have chosen a certain method over another are often left out as one can read the reasons in the respective paper, which I guess follows the principle of omitting unnecessary material.

4. Conclusions

The conclusion is a brief summary of the main findings and includes a discussion, perspective and future work etc.

Verdict: The conclusion doesn’t go into any details anymore, which is good as it should be brief. What I’m missing from it is some sort of critical questioning of the results. What the authors describe is only positive and that they are happy. That’s maybe what truly are. There’s no mention of future work of possible improvements.

5. Acknowledgements

As the name says, in this part the authors acknowledge people that helped the creation of the paper.

6. References

References should be complete and relevant and correctly cited.

7. Appendices

This part is optional and sometimes used to append large figures, tables or code.

Style Review

Graphs, Figures and Tables

If you haven’t heard about Edward Tufte, go read about his work now. He has produced some great literature about statistics visualizations. He talks about information design (e.g. data-ink ratio), criticizes PowerPoint, encourages small multiples (produce many small visualizations instead of one superimposed graph) and other techniques.

Other than that it should be clear that figures and tables should be easy to read and understand, show relevant data and have a caption explaining them.

Verdict: The paper has some very good graphs, and some which are not that good. It seems they have been produced in different styles. Generally I would like the visualizations to have a common style.

I take two examples:

Good: I like the color coding and the display of the seasonal effects of the weekdays. The main thing here to criticize is the color choice for Saturday as it’s barely readable. They could also reduce the grey background lines and increase the data to ink ratio.

Bad: This figure looks less optimal. On the first chart I don’t know what the dotted line is or why the other one is blue. It is also dangerous to put three charts together implying they could be compared even they have different timelines and should not be compared. Also, isn’t weekly also a trend (referring to the y-axis descriptions).

References

Justin Zobel (2014) Writing for Computer Science, Third Edition, New York: Springer.

Facebook's Prophet and the paper

Fri, 20 Mar 2020 22:00:00 GMT

For the AI seminar, I am attending, I selected the following paper:

Sean J. Taylor & Benjamin Letham (2018) “Forecasting at Scale”, The American Statistician.

Abstract

In the coming weeks, I want to evaluate the method described in the paper and see what can be accomplished with it in an environment relevant to me. The relevant environment for me is time series, mostly business time series, but can be of different sorts as well.

Content

The paper “Forecasting at scale” describes the method that Facebook uses to do forecasting on time series data based on an additive model where non-linear trends are fit with yearly, weekly and daily seasonality, plus holiday effects. The procedure is robust to missing data and shifts in the trend, and typically handles outliers well.

Motivation

I’m explaining my motivation behind selecting the paper:

Why have I chosen this paper?

Last year I got a task when doing volunteering service for the country, to work on a time series data set. I built a proof of concept system that could process the gigabytes of CSV files and aggregate the time series to different sample durations. The time series were also transformed using different methods such as CUSUM and EWMA to make them easier to interpret. I visualized them with Grafana that I connected to a self-written Python backend using Flask. The processing of the data was limited to my dev machine and I was restricted from using any other infrastructure. Because it became clear that single-thread processing would not finish the processing within days, I had to parallelize it with Dask, which shortened the processing time dramatically.

After visualizing the immense amount of data it became clear that we will find some seasonality and trend in subsets of the time series. Thus I had the idea to decompose the filtered subsets into seasonality and trend. This gave the opportunity to then create a prediction interval that can be used to find anomalies in the data.

After a quick and dirty implementation, it became clear that producing nice graphs could turn out to be tricky. A colleague stumbled upon the anomalize package in R.

We tried it out but concluded that special holidays, weekends and so on still were a problem for anomalize which uses STL (Seasonality and Trend decomposition using LOESS, Paper) and methods from Twitter’s AnomalyDetection library.

So our research continues and we found the library Prophet which looked promising, as it’s method promises that it can not only cope with holiday effects but also work at scale, has automatic changepoint detection and allows for Analyst-in-the-Loop Modeling.

But unfortunately the time we had run out and I had no chance to look at this method in depth. This AI seminar now gives me the chance to catch up.

Why is it relevant and interesting?

Facebook has developed the Prophet library in house and is sharing it as open source with the community. At my day job, we’re operating applications at scale and thus are confronted with the same problems that Facebook has tried to solve.

I think it can be interesting for larger companies trying to achieve application operating models that can do self-healing, scale automatically using predictions, anticipate surges and find anomalies so an operator can have a closer look and feed the result back into the model. Until 2025 we want to have AIOps established in the company. And we’re certainly not the only company trying to do this.

Expectations & Goals

My goal of working through this paper and the accompanying library is to deepen my knowledge of how time series analysis can be approached beside the methods I already know like STL. Also, I would like to be able to tell our business owners and fellow coworkers if this method could be a useful tool.

AI Seminar Intro

Sun, 08 Mar 2020 12:30:00 GMT

As mentioned in my first blog post New Beginnings, I’m inscribed to a course named “AI Seminar”. In this course, each student is expected to read a paper of their choice. We will go through understanding writing style, understanding content, reviewing and criticizing a paper, evaluating yourself oneself and have a final presentation.

Why I have chosen this seminar

I have not touched on that many results of scientific work. This course is ideal to fix that for me. We will discuss in person and work through a paper in depth. I feel like this is something that will always be handy to have experienced regarding my future career, even if I would never write a paper. But since I’m inscribed in this seminar, I may even would feel comfortable writing a paper.

That the papers we read are supposed to be in the field of AI is also something adding to my motivation to choose this seminar as my specialization is Data Science. Meaning I can learn about some AI methods that I am interested in anyway. So, two flies with one clap.

My learning goals

My learning goals pretty much align with the course description. So I want to learn about scientific writing, what a good paper is about and get familiar with a topic of my interest.

I want to know in detail how the process of writing scientific papers is like and hear from an expert that has experience in this.
I want to try criticizing a paper and get feedback.
I want to refine my writing skills.
I want to get to know the method explained in my chosen paper in detail.
I want to establish a regular blogging habit beyond this seminar.
I want to share my learning process and gain insights how others do doing the same.

New Beginnings

Sun, 01 Mar 2020 22:40:00 GMT

Why am I starting my own blog? This post is the first ever post in a personal blog. I try to explain why having a personal blog is a good idea IMHO and what gives me the motivation to start it now. If you don’t have your own place to write, this maybe gives you some motivation to do so too.

Who am I

I’m a software engineer working at a large telecommunications company in Switzerland. Here I build large scale full stack web applications with technologies such as Springboot and Angular. We pursue a modern development pipeline with state of the art tooling and automation with autonomous releasing and advanced monitoring techniques.

Besides that, I’m doing my Master of Science with specialization in Data Science.

So I have lots of things I learn daily. Some things are personal ideas, some things are implementations of ideas of the industry, some things are just personal reflection.

Motivation #1: Writing down what I learned

If you learn new things on a daily basis, you will eventually have learned that many relevant things you are not able to recall each and every learning correctly unless you write it down.

Now if you write it down for your personal use only, you can write down pretty much anything. It could be confidential, it could be targeting people. This may work very well for you. But if there is a greater learning, and you stay on the detail level about the problem, you may not be able to generalize the learning to new problems and you have not thought enough about it to abstract the problem and solution.

If you write a blog that can be accesses publicly, you need to remove any confidential information from it. What is left at that point is an abstract description of problem and solution that in theory should be applicable to you in the future in a new setting. Because you’ll not always be in the same environment forever and your problem and solution space changes.

Also while you have go through the abstraction process you start to get to the core of you learning.

You might learn useful things that you share verbally, by email or text message, but with a blog you can share what you learned with people you don’t know. They would probably reach a single article of the blog through a Google search, read it and apply it to their problem.

This is different to Stackoverflow or GitHub issues that you write about a problem you had with your own motivation. You’re not solving somebodies problem that they posted and want an answer for.

I also think that the blog post would be more of a guide to a larger problem with an introduction and conclusion, several sections and references. It often is not just a few sentences and a code snippet.

By sharing the blog post publicly and including a discussion or comment section, you are able to get feedback. This may be that you explained something not clearly or that there are other people that have thought about this problem before that you don’t know about.

Motivation #3: Refine my writing skills

Explaining something so everybody can understand is an essential skill for engineers working in teams. If you want to introduce a new concept, raise awareness about architectural problems and so on, not everybody will understand what you mean without investing time explaining it.

By writing a blog, you are forced to explain complex settings in a clear way. I think the only real way to get better at this, is to exercise it. So a blog can be an excellent tool to refine your writing skills.

Also my mother tongue is not English, so this blog allows me to refine my English writing.

Motivation #4: Trying Gatsby.js

I’m reading a lot of people are using Gatsby.js. It sounds like a promising technology and I am eager to get my hands on it. It has some cool concepts. So, I’m building this blog with Gatsby.js.

See https://github.com/sebastianhaeni/blog if you want to read the source code.

Motivation #5: AI Seminar course

Lastly, and where I finally got the motivation to start this blog is a seminar about AI that I’m inscribed in for my Masters. In that course, we go through a scientific paper and are expected to write blog posts about the process and what we learned when reading and asking questions about it.

Blog of Sebastian Häni

What do Architects do?

What is Architecture?

Getting rid of Software Architecture

What do Architects do?

Implications

Principles

Architect’s Activities

How can an architect do that?

Approach

Minimal Viable Architecture

References

Architectural Significance

Short Form:

Long Form

Decentralized Architecture

Decision-making via the “Advice Process”

The Four Supporting Elements

Architectural Decision Records (ADRs)

Architecture Advisory Forum

Team-sourced Architectural Principles

Tech Radar

Conclusion

Maven Dependency Tree

Example

Introduction to Distributed Tracing

Distributed Tracing

Tools

System Architecture

OpenTelemetry Introduction

What is OpenTelemetry

The need for standards

Instrumentation

Collection

Upcoming

Setting up Apache Spark and Zeppelin

What is Spark and Zeppelin

Setup

Instructions

Reverse Proxy

Spark and Zeppelin

Base Image

Apache Spark

Apache Zeppelin

Compatibility Issues

Orgazining optimized code reviews in your team

Contents

Change Reviews

Review Planning

How to define the reviewer?

Code Review is always required

Review Execution

Review Handover

Review Feedback

Review Contents

Review Completion

Merge

Follow-Ups

PO Review

UX Review

Contribution guidelines

Commit message template

Branching Model

Feature Branches

Docs Branches

How we use JIRA

Before a sprint

During a sprint

PO Review

Tech Discussions

Conclusion

Seminar Assessment on Oneself

Goal Verification

1. I want to know in detail how the process of writing scientific papers is like and hear from an expert that has experience in this.

2. I want to try criticizing a paper and get feedback.

3. I want to refine my writing skills.

4. I want to get to know the method explained in my chosen paper in detail.

5. I want to establish a regular blogging habit beyond this seminar.

6. I want to share my learning process and gain insights how others do the same.

Comparing myself to others