Software Engineering

week 1

1. intro, software architecture
2. software development life cycles

readings

• Software Architecture Metaphors - by Lisa Stähli
• What is Software Architecture? - by Kevlin Henney
• Practical Tips on Software Architecture Design, Part One - by Marco Bürckel
• Practical Tips on Software Architecture Design, Part Two - by Marco Bürckel
• Why Write ADRs - by Eli Perkins

week 2

1. week 1 summary
2. scrum
3. kanban

readings

scrum

• The Scrum Guide

zombie scrum

• Zombie Scrum - Symptoms, Causes and Treatment
• The Rise Of Zombie Scrum

kanban

• the Kanban Guide

week 3

1. week 2 summary
2. requirement analysis
3. user story mapping
4. project assignment

readings

requirement analysis

• SDLC Guide: Requirement Analysis in Software Engineering - by Arkadiusz Krysik
• What’s in a Story? - by Daniel Terhorst-North
• Making sense of MVP (Minimum Viable Product) – and why I prefer Earliest Testable/Usable/Lovable - by Henrik Kniberg
• How to Carry Out a Requirements Analysis - by Kelechi Udoagwu

user story mapping

• Jeff Patton’s original blogpost
  • Five common story mapping mistakes - by Jeff Patton
• The Ultimate Guide to User Story Mapping - by Nick Muldoon
• Mapping User Stories in Agile - by Anna Kaley
• Quickstart Guide to User Story Mapping

optional

• What is a Data Flow Diagram
• Mastering Python’s Hidden Power: Monad Design Patterns for Smarter Code

week 4

1. week 3 summary
2. UML
3. C4

readings

UML

• What is UML 2?
• Unified Modeling Language (UML) Diagrams

C4 model

• C4 model
• The C4 Model for Software Architecture - by Simon Brown

optional

• What is object-oriented programming (OOP)? Explained in depth

week 5

1. week 4 summary
2. design patterns

readings

• The Full-stack Software Design & Architecture Map - by Khalil Stemmler

SOLID

• A Solid Guide to SOLID Principles - by Sam Millington
• SOLID Design Principles - Devopedia

inversion of control

• Inversion Of Control - by Martin Fowler
• Don’t Call Us, We’ll Call You: The Hollywood Principle for Maintainable Code - by Teni Gada

topologies

• Hexagonal Architecture - by Sven Woltmann
• The Onion Architecture - by Jeffrey Palermo

MVC-MVP-MVVM

• MVC vs MVP vs MVVM - by Priya Pedamkar
• Alternatives To MVC - by Anthony Ferrara

optional

• The Composition Over Inheritance Principle
  • with Python examples
• Railway oriented programming - by Scott Wlaschin

week 6

1. week 5 summary
2. interfaces
3. implementation planning

readings

• riskstorming.com
• OWASP Risk Rating Methodology
• Story points and estimation - by Dan Radigan
• What Scrum Says About Estimates
• How to create a Minimal, Reproducible Example

week 7

1. week 6 summary
2. wireframing
3. clean code

readings

• What is wireframing? A complete guide - by Louise Bruton
• UI Prototypes
• Practical training for creating more usable products - by balsamiq

week 9

1. week 7 summary
2. code quality
3. code review

readings

• Code Smells - by Jeff Atwood
• How To Write Unmaintainable Code - by Roedy Green
• Code Review Guidelines for Humans - by Philipp Hauer
• How to Do Code Reviews Like a Human (Part One) - by Michael Lynch
  • Part Two

week 10

1. week 9 summary
2. testing

readings

• What Is Test-driven Development and Why It’s Important - by Anna Khrupa
• What’s in a Story? - by Daniel Terhorst-North
• Modern Test-Driven Development in Python - by Jan Giacomelli
• The Cycles of TDD - by Robert C. Martin
• Arrange-Act-Assert
• Readme Driven Development - by Tom Preston-Werner
• Defining Legacy Code - by Eli Lopian

optional

• The Transformation Priority Premise - by Robert C. Martin
• There is No Such Thing as a Unit Test - by Andrew Watson
• Why Most Unit Testing is Waste - by James O Coplien

week 11

1. week 10 summary
2. automatization

readings

• Version control concepts and best practices - by Michael Ernst
• Continuous Integration - by Martin Fowler
• What Is Software Deployment? Process and Best Practices - by Washija Kazim
• 11 DevOps Principles and Practices to Master: Pro Advice - by Fernando Doglio
• The Cost of Interruption for Software Developers - by Steven To
• The high price of context switching for developers & ways to avoid it - by Nitin Pande
• Programmer Interrupted - by Chris Parnin

optional

• A successful Git branching model - by Vincent Driessen
  • a criticism: A succesful Git branching model considered harmful - by Jussi Judin
• a comparison of branching strategies - by Scott Shipp
• Why I don’t like SemVer anymore - by Brett Cannon

week 12

1. summary
2. mindmap
3. mock exam

schedule

materials

tools

• diagram drawing:
  • draw.io
  • Google Drawings
  • plantuml
• whiteboard:
  • excalidraw
• kanban board
  • Trello
• code hosting / task management
  • GitHub

program vs. software

program is like a recipe

for i in range(1, 101):
    if i % 15 == 0:
        print("" + "FizzBuzz")
    elif i % 3 == 0:
        print("" + "Fizz")
    elif i % 5 == 0:
        print("" + "Buzz")
    else:
        print(i)

programming vs. software development

• does that mean a program is not
  • planned
  • documented
  • tested
  • verified?

• the main difference is the formality of the process
  • which correlates the complexity of the project

software development is like building a house

• the software development is often compared to house building
  • which is more like a sequential process
• after the planning (including building permit, budget, etc.), the foundation is built first, then walls and the roof
  • these phases cannot be swapped
• after the construction is finished, the contractor leaves the site

software development not is like building a house

a software does not have to obey the laws of physics

• in software development you can start with the door of the second floor bathroom
• the size of a room can be changed during the construction – even several times

software development is like gardening

what is software rot?

dormant rot: the software in question is not changed, but as the environment evolves, it eventually becomes dysfunctional

active rot

• the software has undergone constant modifications but gradually loses its integrity
• the constant updates / bug fixing can lead to an evolution process,
  • which makes the program deviate from its original design,
  • even introducing newer bugs

software development is like gardening - cont.

software craftmanship

• Not only working software, but also well-crafted software
• Not only responding to change, but also steadily adding value
• Not only individuals and interactions, but also a community of professionals
• Not only customer collaboration, but also productive partnerships

software growth

24 million lines of code – operational and support – needed for the F-35 to be fully operational

the more, the better?

if we wish to count lines of code, we should not regard them as “lines produced” but as “lines spent”

E. W. Dijkstra EWD 1036

keep it simple

A designer knows he has achieved perfection not when there is nothing left to add, but when there is nothing left to take away.

– Antoine de Saint-Exupéry

Linux 5.8 – 800,000 new lines of code

• how is it manageable?
  • process
  • version control
• each change must do only one thing
  • proper documentation
• changes cannot break the software
  • rigorous and automated testing

version control

• version control (a.k.a. revision control) is system for recording and managing changes made in files
• commonly used to manage source code
  • however, it can be used to tracking changes to any kind of files
• people often employ their own version control system, without realising it

why you should use version control (for everything)

project complexity

what is software architecture?

“Architecture” is a term that lots of people try to define, with little agreement. There are two common elements: One is the highest-level breakdown of a system into its parts; the other, decisions that are hard to change.

– Martin Fowler - Patterns of Enterprise Application Architecture

In most successful software projects, the expert developers working on that project have a shared understanding of the system design. This shared understanding is called ‘architecture’. This understanding includes how the system is divided into components and how the components interact through interfaces. These components are usually composed of smaller components, but the architecture only includes the components and interfaces that are understood by all the developers.

Ralph Johnson, XP mailing list

All architecture is design but not all design is architecture. Architecture represents the significant design decisions that shape a system, where significant is measured by cost of change.

– Grady Booch

topologies

Layered Architechture

• Layered Architecture: Introduction
• Layered Architecture: Component Interactions

message bus

• shared communication channel that connects multiple components or services
• simple, extensible

server/client architecture

• consists of two parts
  • client and server
• distributed
• always the client initiates a connection to the server
• while the server process always waits for requests from any client

references

content

• software development life cycle
• waterfal
• V model
• incremental
• agile
  • SCRUM
• kanban

waterfall model

criticism

• Clients may not know exactly what their requirements are before they see working software and so change their requirements, leading to redesign, redevelopment, and retesting, and increased costs. (Parnas & Clements, 1986) In most cases the people who comission the building of the a software system do not know exactly what they want and are unable to tell us all they know.

modified waterfall model

V model (Forsberg & Mooz, 1991)

• still rigid
• each phase has output and a review process
  • errors are found at early stage
  • decreases the risk of failure
• large to small: testing is done in a hierarchical perspective

iterative model

• software is built incrementally,
  • with each iteration adding new features or refining existing ones
• possible to get feedback after each iteration
• can be rigid within an iteration

agile model

• continuous collaboration and fast response to change, while the iterative model takes a more gradual approach, building up the final product over multiple iterations
• scrum is an agile methodology

the agile manifesto

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

agilemanifesto.org

. . .

Principles behind the Agile Manifesto

1. Principle behind the Agile Manifesto

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

. . .

Release early. Release often. And listen to your customers.

Eric S. Raymond: The Cathedral and the Bazaar (1997)

2. Principle behind the Agile Manifesto

Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.

Principles behind the Agile Manifesto

3. Principle behind the Agile Manifesto

Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter timescale.

4. Principle behind the Agile Manifesto

Business people and developers must work together daily throughout the project.

5. Principle behind the Agile Manifesto

Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.

6. Principle behind the Agile Manifesto

The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.

7. Principle behind the Agile Manifesto

Working software is the primary measure of progress.

8. Principle behind the Agile Manifesto

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

9. Principle behind the Agile Manifesto

Continuous attention to technical excellence and good design enhances agility.

10. Principle behind the Agile Manifesto

Simplicity–the art of maximizing the amount of work not done–is essential.

11. Principle behind the Agile Manifesto

The best architectures, requirements, and designs emerge from self-organizing teams.

12. Principle behind the Agile Manifesto

At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.

manifesto for software craftsmanship

agile vs. waterfall

agile vs. waterfall

Can waterfall work? Yes, it can, if the customer exactly knows what they want and can express it in technical terms.

Although usually the customer does not know what they want, so agile usually work better.

SCRUM

kanban

• notes move from left to right
• order denote priority
• allways process the right- and topmost one to finish it ASAP
• tool-dependent but a note can indicate:
  • who’s responsible
  • how much effort to do it
  • etc.

kanban for job hunting

kanban in software development

SCUM + kanban = scrumban

• SCRUM and kanban can coexist
• quite new methodology

references

this presentation is based on The Scrum Guide (2020)
by Ken Schwaber and Jeff Sutherland (Schwaber & Sutherland, 2020)

available from scrumguides.org under CC BY-SA 4.0

or download directly from here

what is scrum?

Scrum is a lightweight framework that helps people, teams and organizations generate value through adaptive solutions for complex problems.

Scrum employs an iterative, incremental approach to optimize predictability and to control risk. Scrum engages groups of people who collectively have all the skills and expertise to do the work and share or acquire such skills as needed.

scrum in a nutshell

scrum team

• the scrum team consists of
  • one scrum master,
  • one product owner,
  • and developers
• there are no sub-teams or hierarchies

size of a scrum team

• small enough to remain nimble and large enough to complete work in a sprint
  • typically 10 or fewer people
  • 3 to 9 people is optimal
  • smaller teams communicate better and are more productive
• if a team becomes too large, they should consider reorganizing into multiple cohesive scrum teams
  • each focused on the same product
  • therefore, they should share the same product goal, product backlog, and product owner

communication within a team

$$ \frac{3(3 − 1)}{2} = 3 $$

$$ \frac{4(4 − 1)}{2} = 6 $$

$$ \frac{5(5 − 1)}{2} = 10 $$

$$ \frac{n(n − 1)}{2} $$

team size

scrum teams are cross-functional

• the members have all the skills necessary to create value each sprint
• they are also self-managing, meaning they internally decide who does what, when, and how

developers

definition of done

product owner

• accountable for maximizing the value of the product resulting from the work of the scrum team
• also accountable for effective product backlog management, which includes:
  • developing and explicitly communicating the product goal,
  • creating and clearly communicating product backlog items,
  • ordering product backlog items, and
  • ensuring that the product backlog is transparent, visible and understood

The product owner may do the above work or may delegate the responsibility to others. Regardless, the product owner remains accountable.

scrum master

• accountable for
  • establishing scrum as defined in the Scrum Guide
    • by helping everyone understand the theory and practice, both within the scrum team and the organization
  • for the scrum team’s effectiveness
    • by enabling the scrum team to improve its practices, within the scrum framework

scrum master serves the scrum team

• coaching the team members in self-management and cross-functionality,
• helping the scrum team focus on creating high-value increments that meet the definition of done,
• causing the removal of impediments to the scrum team’s progress, and
• ensuring that all scrum events take place and are positive, productive, and kept within the timebox

scrum master serves the product owner

• helping find techniques for effective product goal definition and product backlog management,
• helping the scrum team understand the need for clear and concise product backlog items,
• helping establish empirical product planning for a complex environment, and
• facilitating stakeholder collaboration as requested or needed

scrum master serves the organization

• leading, training, and coaching the organization in its scrum adoption,
• planning and advising scrum implementations within the organization,
• helping employees and stakeholders understand and enact an empirical approach for complex work, and
• removing barriers between stakeholders and scrum teams

scrum events

sprint

Sprints are the heartbeat of Scrum, where ideas are turned into value.

• fixed length events (maximum one month) to create consistency
• a new sprint starts immediately after the end of the previous

during the sprint

• no changes are made that would endanger the sprint goal,
• quality does not decrease,
• the Product Backlog is refined as needed, and
• scope may be clarified and renegotiated with the product owner as more is learned

about sprints

• in the case of a long sprint
  • the sprint goal may become invalid
  • complexity may rise, and
  • risk may increase
• each sprint may be considered a short project
• a sprint could be cancelled if the sprint goal becomes obsolete
• only the product owner has the authority to cancel the sprint

tracking progress - burndown chart

burndown chart is a graphical representation of work left to do versus time (Wikipedia contributors, 2024)

• two week sprint
• 26 tasks
• ideal work line (green)

• actual work line (red)
  • not realistic
• finished tasks per day (blue bars)

tracking progress - cumulative flow diagram

sprint planning

topics of sprint planning - why is this sprint valuable?

• the product owner proposes how the product could increase its value and utility in the current sprint
• the whole scrum team then collaborates to define a sprint goal that communicates why the sprint is valuable to stakeholders
• the sprint goal must be finalized prior to the end of sprint planning

topics of sprint planning - what can be done this sprint?

• in agreement with the product owner, the developers select items from the product backlog to include in the current sprint agenda
• the scrum team may refine these items during this process
• not easy to select how much can be completed within a sprint
• the more the developers know about
  • their past performance (e.g., burndown charts),
  • their upcoming capacity (e.g., vacation, holidays), and
  • the definition of done,
  • the more confident they will be in their sprint forecasts

topics of sprint planning - wow will the chosen work get done?

• for each selected item, the developers plan the work necessary to create an increment that meets the DoD
• often done by decomposing product backlog items into smaller work items of one day or less
  • tasks completable in 1-4 hours may be preferred
• how this is done is up to the developers
  • no one else tells them how to turn product backlog items into increments

sprint planning - summary

1. select the sprint goal
2. select product backlog items to achieve the sprint goal
3. plan how they shall be implemented

The sprint goal, the product backlog items selected for the sprint, plus the plan for delivering them are together referred to as the sprint backlog.

daily scrum

• the purpose of the daily scrum is to inspect progress toward the sprint goal and adapt the sprint backlog as necessary, and discussing the upcoming planned work
• the daily scrum is a 15-minute event for the developers
• to reduce complexity, it is held at the same time and place every working day of the sprint
• if the product owner or scrum master are actively working on items in the sprint backlog, they participate as developers

daily stand-up

• the developers can select whatever structure and techniques they want
• also called daily stand-up, because they stand up from the desks and go to a meeting place
  • as it is limited to 15 minute, no need to book a meeting room;
• daily scrums improve communications, identify impediments, promote quick decision-making, and consequently eliminate the need for other meetings

three questions

keep daily stand-up short

• answer the three question only
  • do not start finding solutions for the problems
• the daily scrum is not the only time developers are allowed to discuss the sprint backlog
• they can meet throughout the day for more detailed discussions

sprint review

• the scrum team presents the their work to key stakeholders
  • and the progress toward the product goal is discussed
• the attendees collaborate on
  what to do next
• the product backlog may also be adjusted to meet new opportunities

• should not limiting it to a presentation, the working product should be demonstrated and discussed
• timeboxed to a maximum of four hours for a one-month sprint
  • for shorter sprints, the event is usually shorter

sprint retrospective

• he purpose of the sprint retrospective is to increase quality and effectiveness
• the sprint retrospective concludes the sprint
• it is timeboxed to a maximum of three hours for a one-month sprint
  • for shorter sprints, the event is usually shorter

retrospective starfish

scrum artifacts

• scrum artifacts represent work or value
• they are designed to maximize transparency of information
• the product backlog
  • progress towards the product goal
• sprint backlog
  • progress within the sprint goal
• definition of done
  • state of the product increment

product backlog

• ordered list of what is needed to improve the product
• describes a future state of the product
• product owner is responsible for its content and prioritization

product backlog refinement

product backlog refinement is the act of breaking down and further defining product backlog items into smaller more precise items.

sprint backlog

increment

• an increment is a step toward the product goal
• each increment is additive to all prior increments
  • and verified, ensuring that all increments work together
• in order to provide value, the increment must be usable
• multiple increments may be created within a sprint
• work cannot be considered part of an increment unless it meets the definition of done

scrum of scrums

zombie scrum

mindset

working software

what is work

treatments

references

this presentation is based on The Kanban Guide (2020) by Daniel S. Vacanti

available from kanbanguides.org under CC BY-SA 4.0

or download directly from here

what is kanban?

principles of kanban

defining and visualizing the workflow

defining and visualizing the workflow

defining and visualizing the workflow

Service Level Expectation

Service Level Expectation

improving the Workflow

• it is common practice to review the DoW from time to time to discuss and implement any changes needed
  • e.g., need a new column for a new state
• not necessary to wait for a formal meeting at a regular cadence to make these changes

kanban measures

Toyota’s six rules (Wikipedia contributors, 2024)

1. Each process issues requests (kanban) to its suppliers when it consumes its supplies.
2. Each process produces according to the quantity and sequence of incoming requests.
3. No items are made or transported without a request.
4. The request associated with an item is always attached to it.
5. Processes must not send out defective items, to ensure that the finished products will be defect-free.
6. Limiting the number of pending requests makes the process more sensitive and reveals inefficiencies.

scrumban

references

requirement analysis

Requirement analysis is all about understanding what software is supposed to do and the constraints it must operate within (Krysik, 2023).

• first step of the software development life cycle (SDLC)
• about understanding the task to avoid costly mistakes

steps of requirement analysis

1. stakeholder identification
2. elicitation of requirements (gathering data)
3. documentation of requirements
4. analysis and negotiation
5. validation and verification

1. stakeholder identification

• not just the customers who commission the software
• also end users
  • different roles / groups

2. elicitation of requirements

the team actively gathers detailed information about what the software needs to do from the identified stakeholders (Krysik, 2023)

3. documentation of requirements

4. analysis and negotiation

5. validation and verification

requirement smells

• based on the idea of code smells (later in the course)
• (language based) signs in the requirements that are not necessarily wrong but could be problematic, e.g.,:
  • subjective language
    • “The architecture as well as the programming must ensure a simple and efficient maintainability.”
  • ambiguous adverbs and adjectives
    • “If the (…) quality is too low, a fault must be written to the error memory.”
  • non-verifiable terms
    • “The system may only be activated, if all required sensors (…) work with sufficient measurement accuracy.”

requirement analysis document example

functional and non-functional requirements

more examples on Wikipedia

dependencies

dependencies within a software

• dependencies between the software components are not always obvious
  • and change over time
• the core functionalities should be determined at the start of the project
  • that can serve as a foundation for the rest of the software
  • this requires comprehensive understanding of the project
    • requirement analysis

minimum viable product

a new product or service is created with the minimum features necessary to satisfy early adopters and gather feedback for future development

minimum viable product

social media platform - example

social media platform - example

social media platform - example

social media platform - example

what is a user story?

behaviour-driven development

• BDD is an extension of Test-Driven Development
  • later in the course
• using behaviour-driven development (BDD) can help you to turn an idea for a requirement into implemented, tested, production-ready code,
  • as long as the requirement is specific enough that everyone knows what’s going on (Terhorst-North, 2007)
• BDD starts from a user story and focuses on adding the acceptance criteria

the structure of a story

Title (one line describing the story)

Narrative:
As a [role]
I want [feature]
So that [benefit]

Acceptance Criteria: (presented as Scenarios)

Scenario 1: Title
Given [context]
  And [some more context]...
When  [event]
Then  [outcome]
  And [another outcome]...

Scenario 2: ...

ATM example

Story: Account Holder withdraws cash

As an Account Holder
I want to withdraw cash from an ATM
So that I can get money when the bank is closed

Scenario 1: Account has sufficient funds
Given the account balance is $100
 And the card is valid
 And the machine contains enough money
When the Account Holder requests $20
Then the ATM should dispense $20
 And the account balance should be $80
 And the card should be returned

Scenario 2: Account has insufficient funds

Scenario 3: Card has been disabled

Scenario 4: The ATM has insufficient funds

it may be difficult to extract knowledge

• the scenarios (and tests) may require exact thresholds
• need to interview domain specialists
• communication barrier, lack of common dictionary

benefits of requirements analysis

• clear project scope and objectives
• improved stakeholder satisfaction
• reduced development costs and time
• enhanced product quality
• better risk management
• facilitates prioritization
• improved communication and collaboration

requirements analysis techniques

• user stories and user story mapping
• gantt charts
• flowcharts
• data flow diagram
• etc.

gantt chart

• project management tool
• illustrates a project schedule
• practical to construct the schedule from the deadline and go backward

flowchart

flowchart example

references

the problem

• the backlog difficult to prioritize
  • identify dependencies
  • already at this point the tasks need detailed understanding
• the backlog is one dimensional
  • priority

user story map

activity

As a social media platform user
I want to follow users
so I can keep up with their posts.

user task

user story mapping

• popularized by Jeff Patton
  • original blog post
  • User Story Mapping, O’Reilly, 2014, ISBN-13: 978-1491904909
• performed in workshops including
  • users,
  • (UI) designers,
  • developers,
  • testers,
  • and stakeholders
• build a shared understanding of the product and a common language

user story map as a document

• not final, not set in stone
• it is possible and encouraged to adjust
  as the more knowledge is acquired about the software
  • versioning
• you are free to move notes up and down
  • change the role (activity ↔ user task)

backbone, skeleton, ribs

iterations

social media platform - example

user story mapping mistakes

programming paradigms

• structural
• procedural
• object oriented

structural

a = 4.2
a = a * 10

if a > 17
    a -= 5
else:
    a += 5

for i in range(10):
    print(i)

for (var i = 0; i < 10; i++) {
    console.log(i);
}

procedural

• extends structural with procedures
  • a.k.a. functions, subroutines
• the two main concepts
  • modularity: organizing the parts of a program into separate modules
    • reusability
  • scoping
    • limit the scope of the variables

procedural - example

def power(a, b):
    r = a
    for _ in range(b - 1):
        r *= r
    return r

a = 3
a = power(a, 3)

object oriented programming

• extends procedural programming with the concept of objects
• main properties of OOP
  1. abstraction
  2. encapsulation
  3. inheritance
  4. polymorphism

1. abstraction

• hiding the complex reality while exposing only the necessary parts
• allows to focus on interactions at a higher level without needing to understand the details of the implementation
• achieved through abstract classes and interfaces, which define a contract for what methods an object must implement without specifying how they should be implemented

2. encapsulation

• bundling data (attributes) and methods (functions) that operate on that data into a single unit known as a class
• this property restricts direct access to some of the object’s components
  • private, public, protected
• can preventing unintended interference and misuse of the methods and data
  • by exposing only the necessary parts of an object through public methods

3. inheritance

• a mechanism that allows one class (subclass or derived class) to inherit attributes and methods from another class (superclass or base class)
• this promotes code reusability, as common functionality can be defined in a base class and reused in derived classes
• results hierarchical relationship which fosters modular design
  • also increases dependency

class-based inheritance

• every object is defined by a class
  • which is a definition or a blueprint
  • describes the structure and behavior of an object
• most common

prototype based inheritance

The object function untangles JavaScript’s constructor pattern, achieving true prototypal inheritance. It takes an old object as a parameter and returns an empty new object that inherits from the old one. If we attempt to obtain a member from the new object, and it lacks that key, then the old object will supply the member. Objects inherit from objects. What could be more object oriented than that?

Douglas Crockford

OO without inheritance

• Go does not support inheritance at all, though it is considered object-oriented
  • at least partially [Go FAQ]
• Bjarne Stroustrup (author of C++) has stated that it is possible to do OOP without inheritance (Stroustrup, 2015)

4. polymorphism

• allows objects to be treated as instances of their parent class
• enables flexibility in code, allowing for methods to perform differently based on the object that invokes them
  • method defined in a base class can be overridden in a derived class to provide specific behavior

Unified Modeling Language

use case diagram

• depicts the interactions between system users (actors) and the system itself
• used to specify the functional requirements
• provides a high-level view
  • helping stakeholders to understand the system’s functionality
• it’s purpose is similar to the user story

elements of the use case diagram

use case diagram - example

class diagram

• describes the structure of a system by its classes
  • their attributes, methods, and the relationships among them
• main building block of the object-oriented modeling

(most common) elements of a class diagram

relations

• association: structural relationship
  • allows one object instance to cause another to perform an action on its behalf
• realization: e.g., class implements a interface
• aggregation: “has a” relation
  • without life cycle control
• composition: stronger form of aggregation
  • where the aggregate controls the lifecycle of the elements

class diagram - example

class diagram - example

object diagram

• special case of a class diagram
• graphical representation of the objects and their relationships
  at a specific moment in time
• provides a snapshot of the system’s structure
• does not show anything architecturally different to class diagram

component diagram

• depicts the component structure and relations
• highlighting the interfaces

state diagram

• a visual representation of the states a system or an object can be in also the transitions between those states
• models the dynamic behavior of the system, capturing how it responds to different events over time
• shows the system’s life cycle

state diagram elements

state diagram - example

activity diagram

• graphical representations of workflows
• similar to flowcharts
  • but uses UML notation
  • and can visualize parallel processing
  • has more features

parallel processing

a join synchronizes two inflows (waits for the slower)

merge after condition

swimlanes

• actions can be separated using “swimlanes”
• swimlanes can represent actors, components or other parts of the software system

sequence diagram

timing diagram

references

what is the issue with UML?

C4 model

• hierarchical set of software architecture diagrams
  • different levels of abstraction for different audience
• has four levels:
  • context, containers, components and code
• popularized by Simon Brown

Level 1: System Context diagram

• high level relation with other systems or users
• similar to use case diagram
• technologies, protocols and other low-level details are not important
• understandable for non-technical people

Shows the software system you are building and how it fits into the world in terms of the people who use it and the other software systems it interacts with.

Simon Brown - The C4 Model for Software Architecture

system context example

notation

Level 2: Container diagram

Zooms into the software system, and shows the containers (applications, data stores, microservices, etc.) that make up that software system. Technology decisions are also a key part of this diagram.

Simon Brown - The C4 Model for Software Architecture

container example

notation

Level 3: Component diagram

Zooms into an individual container to show the components inside it. These components should map to real abstractions (e.g., a grouping of code) in your codebase.

Simon Brown - The C4 Model for Software Architecture

component example

notation

Level 4: Code

Finally, if you really want or need to, you can zoom into an individual component to show how that component is implemented.

Simon Brown - The C4 Model for Software Architecture

code example

additional diagrams in C4 model

• system landscape diagram
  • even broader environment than the system context diagram
• dynamic diagram
  • based on the UML communication and sequence diagrams
• deployment diagram
  • based on the UML deployment diagram

system landscape diagram

dynamic diagram (collaboration style)

dynamic diagram (sequence)

deployment diagram

• shows where an instance of a software system is running e.g.,
  • physical infrastructure (e.g. a physical server or device)
  • virtualised infrastructure,
  • containerised infrastructure (e.g. a Docker container),
  • an execution environment (e.g. a database server, Java EE web/application server, Microsoft IIS), etc.
• deployment nodes can be nested

summary

overview first, zoom and filter, then details on demand

– Ben Shneiderman

suggested software

software design and architecture stack

gang of four (GoF) design patterns

• GoF: Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides
• 23 common software design patterns
  • published in “Design Patterns: Elements of Reusable Object-Oriented Software” (1994) (Gamma et al., 1994)
• provides solutions to common design problems
• categorized into three main groups
  1. creational
  2. structural
  3. behavioral

the 23 (GoF) design patterns

bridge pattern (structural)

GoF design patterns in functional programming

Peter Norvig demonstrated that 16 out of the 23 patterns are simplified or eliminated by language features in Lisp or Dylan (1998) (Norvig, 1998)

more about it from Scott Wlaschin (Wlaschin, 2014)

You aren’t gonna need it (YAGNI)

coupling

• the degree of interdependence between software modules
• coupling is usually contrasted with cohesion
  • low coupling often correlates with high cohesion, and vice versa

SOLID principles

SOLID is a mnemonic acronym for five design principles intended to make object-oriented designs more understandable, flexible, and maintainable (Wikipedia contributors, 2024d)

• single responsibility principle
• open-closed principle
• Liskov substitution principle
• interface segregation principle
• dependency inversion principle

single responsibility principle

a class should do one thing and therefore it should have only a single reason to change

open-closed principle

classes should be open for extension and closed to modification

class Shape:
    pass


class Square(Shape):
    def __init__(self, width: float):
        self.width = width

class Circle(Shape):
    def __init__(self, radius: float):
        self.radius = radius

class AreaCalculator:

    def sum(self, shapes: list[Shape]) -> float:
        result = 0
        for shape in shapes:
            if isinstance(shape, Square):
                result += shape.width**2
            elif isinstance(shape, Circle):
                result += shape.radius**2 * math.pi

        return round(result, 2)

open-closed principle

class Shape:
    pass

class AreaInterface:
    def area(shape: Shape) -> float:
        pass

class Square(Shape, AreaInterface):
    def __init__(self, width: float):
        self.width = width

    def area(self) -> float:
        return self.width**2

class Circle(Shape, AreaInterface):
    def __init__(self, radius: float):
        self.radius = radius

    def area(self) -> float:
        return round(self.radius**2 * math.pi, 2)

class AreaCalculator:
    def sum(self, shapes: list[Shape]) -> float:
        return sum([i.area() for i in shapes])

Liskov substitution principle

if class A is a subtype of class B, B should be able to replaced with A without disrupting the behavior of the program (Millington, 2019)

Liskov substitution principle - example

class Rectangle:

    def __init__(self, width: int, height: int):
        self.__width = width
        self.__height = height

    def setWidth(self, width: int):
        self.__width = width

    def setHeight(self, height: int):
        self.__height = height

    def getWidth(self):
        return self.__width

    def getHeight(self):
        return self.__height

    def getArea(self):
        return self.__width * self.__height

class Square(Rectangle):

    def __init__(self, width: int):
        super().setWidth(width)
        super().setHeight(width)

    def setWidth(self, width: int):
        super().setWidth(width)
        super().setHeight(width)

    def setHeight(self, height: int):
        super().setWidth(height)
        super().setHeight(height)

>>> r = Rectangle(2, 3)
>>> print(r.getArea())
6

>>> s = Square(2)
>>> print(s.getArea())
4

Liskov substitution principle - example

def getAreaTest(r: Rectangle):
    width = r.getWidth()  # width is 2
    r.setHeight(10)
    return f"Expected area of {width * 10}, got {r.getArea()}"

>>> r = Rectangle(2, 3)
>>> print(r.getArea())
6

>>> s = Square(2)
>>> print(s.getArea())
4

>>> print(getAreaTest(r))  # rectangle
Expected area of 20, got 20

>>> print(getAreaTest(s))  # square
Expected area of 20, got 100

this example violates the Liskov substitution principle

interface segregation principle

states that many client-specific interfaces are better than one general-purpose interface. Clients should not be forced to implement a function they do no need.

dependency inversion principle

Dependency inversion principle says that modules should depend upon interfaces or abstract classes, not concrete classes. It’s an inversion because implementations depend upon abstractions and not the other way round. (Millington, 2019)

hollywood principle (inversion of control)

don’t call us, we’ll call you

• for control flow management
• IoC shifts control from the application to an outside framework
• promotes a more modular design by decoupling components
  • however, adding an IoC framework can increase complexity
    • with a significant learning curve for those unfamiliar with the concept
• e.g., Spring Framework, ASP.NET Core

topologies

Object-oriented design (OOD) is the process of planning a system of interacting objects to solve a software problem (Wikipedia contributors, 2024c).

control flow? structure?

server/client architecture

• consists of two parts
  • client and server
• distributed
• always the client initiates a connection to the server
• while the server process always waits for requests from any client

message bus

• shared communication channel that connects multiple components or services
• simple, extensible

message bus types

layered

number of layers in a layered architecture is not set to a specific number

• presentation layer (a.k.a. UI layer, view layer)
  • responsible for user interactions with the software system
• application layer (a.k.a. service layer)
  • aspects related to accomplishing functional requirements
• business (logic) layer
  • responsible for algorithms, and programming components
• data access layer (a.k.a. persistence layer)
  • responsible for handling data, databases

layered - properties

onion architecture

• popularized by Jeffrey Palermo
• code can depend on layers more central, but code cannot depend on layers further out from the core
  • all coupling is toward the center
• the database is not the center, it is external
  • the data model is in focus, whereas in layered data is the foundation
• relies on the dependency inversion principle
• appropriate for long-lived business applications
  • also applications with complex behavior

hexagonal - motivation

• invented by Alistair Cockburn (Cockburn, 2010)
• application should be equally controllable by users, other applications, or automated tests
  • for the business logic, it makes no difference whether it is invoked from a user interface, a REST API, or a test framework
• infrastructure modernization should be possible without changing the business logic

hexagonal (ports & adapters)

advantages

• modifiability
• isolates responsibilities
• once the ports are defined, the work on the components can be divided among developers

disadvantages

• the effort of port-adapter implementation is non-negligible
• for smaller applications, the extra effort is not worth it

• hexagonal architecture does not specify what is inside the application hexagon
• represents a single design decision:
  • wrap your application in an API and put tests around it

hexagonal vs. layered

Model-View-Controller (Wikipedia contributors, 2024b)

• architectural pattern
• MVC pattern was implemented as early as 1974 in the Smalltalk project

• view is responsible for rendering UI
• controller responds to the user input and performs interactions on the data model
• model is responsible for managing the data

• the view and the model are tightly coupled
• view is monolithic and usually couples tightly with the UI framework
  • unit testing the view becomes difficult

MVC - MVP - MVVM

ASP.NET, Django (Python), Ruby on Rails, Laravel (PHP)

Windows Forms, Java Swing

WPF, AngularJS

user statistics example

as a user I want to see my activity to see my progress

display user statistics including

• username
• profile image
• registration date
• progress in course
• daily activity in the current month

architecture v1

architecture v1 - class

in this case the UI has to calculate the daily activity

• tight coupling
• single responsibility principle violated

architecture v2

architecture v2 - class

data collector still has the whole user data but that aligns with its purpose

data aggregator calculates everything and the UI only displays it

architecture v2.1 - class

UI might be on a client

different code base, different language

architecture v3

architecture v3 - SQL

SELECT
    CAST(strftime('%W', timestamp) AS INTEGER) AS week_of_year,
    CAST(strftime('%u', timestamp) AS INTEGER) AS day_of_week,
    count(*) AS count
FROM activity
WHERE
    user_id = 42 AND
    week_of_year > 35 AND
    week_of_year < 40
GROUP BY
    week_of_year,
    day_of_week
;

architecture v3 - SQL

SELECT
    lesson / 50.0 AS progress
FROM activity
WHERE
    user_id = 42 AND
    result = 'success'
ORDER BY
    lesson DESC
LIMIT 1;

architecture v3 - issues

• hard dependency on database
  • business logic in persistence layer
  • code depends on the SQL dialect
    • can be mitigated with an object-relational mapping (ORM) framework but that would also be a dependency
• may not suitable for complex aggregations
  • stored functions just increase dependency
• harder to unit test

record architecture decisions

# Title

## Status

What is the status, such as proposed, accepted, rejected, deprecated, superseded, etc.?

## Context

What is the issue that we're seeing that is motivating this decision or change?

## Decision

What is the change that we're proposing and/or doing?

## Consequences

What becomes easier or more difficult to do because of this change?

why write ARDs?

references

an interface is a shared boundary across which two or more separate components of a computer system (Wikipedia contributors, 2024a)

interface is an agreement

user statistics example - interfaces

user statistics - C4 component

interface mocking

• an interface is a boundary where a module can be separated
• in the user statistics example there are two parts: a backend and a frontend
• with a well defined interface, the frontend can work regardless of the backends’s state
  • e.g., using a mock backend

user statistics - mock backend

require 'sinatra'

def generate_progress
  rand.round(2)
end

def generate_activity_matrix
  result = []
  (1..4).each do |_w|
    daily = []
    (1..7).each do |_d|
      daily.push rand(10)
    end
    result.push daily
  end
  result
end

get '/user-statistics' do
  data = {}
  data['name'] = 'Marvin'
  data['id'] = 42
  data['registration'] = '2019-10-02'
  data['progress'] = generate_progress
  data['activity'] = generate_activity_matrix
  return data.to_json
end

{
    "name": "Marvin",
    "id": 42,
    "registration": "2019-10-02",
    "progress": 0.92,
    "activity": [
        [4,9,7,4,7,1,8],
        [9,8,1,8,4,1,7],
        [3,6,8,4,2,4,5],
        [3,5,5,3,2,9,7]
    ]
}

frontend development

a mock backend should be enough for a frontend developer to create and test the user statistics view of the user interface

{
    "name": "Marvin",
    "id": 42,
    "registration": "2019-10-02",
    "progress": 0.92,
    "activity": [
        [4,9,7,4,7,1,8],
        [9,8,1,8,4,1,7],
        [3,6,8,4,2,4,5],
        [3,5,5,3,2,9,7]
    ]
}

it may be presented to the customer

fast feedback, agile, and so on…

do not change the interface (without notice)

do no break the userland

the number one rule of kernel development is that “we don’t break users”

– Linus Torvalds

API changes should be communicated

API versions

• 2021-06-30: API v1’s end of live
  • service does not accept connections via APIv1
  • code can be removed (no need to maintain it anymore)

language level

from shapely import Polygon
import geopandas as gpd

p1 = Polygon([[1, 2], [3, 2], [3, 4], [1, 4]])
p2 = Polygon([[2, 3], [4, 3], [4, 5], [2, 5]])

gpd.GeoDataFrame(geometry=[p1, p2]).unary_union

DeprecationWarning: The ‘unary_union’ attribute is deprecated, use the ‘union_all()’ method instead.

def unary_union(self):
    warnings.warn(
        "The 'unary_union' attribute is deprecated, "
        "use the 'union_all' method instead.",
        DeprecationWarning,
        stacklevel=2,
    )
    return self.union_all()

java

public class Worker {
    /**
     * Calculate period between versions
     * @deprecated
     * This method is no longer acceptable to compute time between versions.
     * <p> Use {@link Utils#calculatePeriod(Machine)} instead.
     *
     * @param machine instance
     * @return computed time
     */
    @Deprecated(since = "4.5", forRemoval = true)
    public int calculate(Machine machine) {
        return machine.exportVersions().size() * 10;
    }
}

IDEs can parse the deprecation decorators and show to the developer during work

a 327 Million Dollar interface miscommunication

NASA and Lockheed Martin mixed up units for the Mars Climate Orbiter (1999)

• spacecraft sent values back to Earth in Newton seconds
• the software in the ground station read those results as pound seconds
  • the guidance and navigation teams was off by a factor of 4.45 times
• this led to a miscalculated trajectory, which doomed the probe and the mission

references

implementation planning

1. define goals
   • practically done with requirement analysis
2. conduct research
   • partially done at the requirement elicitation phase
3. map out risks
4. schedule milestones
5. assign responsibilities and tasks
6. allocate resources

conduct research

learning could be a task

experiments

• sometimes the task requires a step that you don’t want to implement (again) but use an existing framework instead
  • e.g., crop a profile picture to a standard size, while keeping the face in focus
• then you may want to search for a suitable framework and do some research about how to use it

fail fast

~ indirect proof, proof of contradiction

• the requirements are known
• try to eliminate candidates as soon as possible
  • unmaintained
  • cannot handle certain file formats
  • target language / version is not supported
• then move on the next one
• document findings (~ decision records)

minimal workable example

identify risks

a risk is a possibility that something bad can happen

• there is risk inherent with building any piece of software
• whether you’re building a completely new greenfield project,
• or adding a new feature to an existing codebase
  • other parts cease to work
  • the new feature alienate users
  • data loss

prioritize risks

• often difficult to prioritise which risks you should take care of
• one way to do this is to map risks to a matrix
• where you evaluate
  • the probability: how likely is it that the risk will happen?
  • and the impact: what is the negative impact if the risk does occur?

risk register

• a risk register is a document used as a risk management tool
• contains all identified risks with additional information
  • e.g., nature of the risk, probability, impact, reference and owner, mitigation measures
• it can be displayed as a table or as a scatterplot

Rust-GCC example

common columns in table-based risk registers:
category, name, responsible, probability, impact, mitigation, action by, action when

risk storming

• visual and collaborative risk identification technique
• created by Simon Brown
  • author of C4 model
• motivation: often only one person evaluated risks
  • ~ four eyes see more than two
• risk evaluation should be collaborative activity

steps of risk storming

1. draw some software architecture diagrams
   • to show what you’re planning to build, at different levels of abstraction
   • ideally C4
2. identify the risks individually
   • gather people in front of the diagrams,
   • ask them to identify what they personally perceive to be risky
   • write a summary of each risk on a separate sticky note,
   • colour coded to represent low, medium, and high priority risks
   • timebox this exercise (e.g. 10 minutes),
   • do it in silence

steps of risk storming

3. converge the risks on the diagrams
   • ask everyone to place their sticky notes onto the diagrams,
   • sticking them in close to the area where the risk has been identified
4. review and summarise the risks
   • review and summarise the output,
   • especially focusing on risks that only one person identified,
   • or risks where multiple people disgree on the priority

mitigating risks

• risks are identified
  • and prioritized
• come up with mitigation strategies
  • either to prevent the risks from happening
  • or to take corrective action if the risk does occur
• focus on the highest priority ones first

mitigation strategies

• education
  • train the team,
  • or hire new team members in areas where you lack experience
• writing code
  • create prototypes, proofs of concept, walking skeletons, etc. to mitigate technical risks by proving that something does or doesn’t work
• re-work
  • change software architecture to remove or reduce the probability/impact of identified risks
    • e.g. removing single points of failure,
  • upon change you should re-run the risk-storming process

walking skeleton

A Walking Skeleton is a tiny implementation of the system that performs a small end-to-end function. It need not use the final architecture, but it should link together the main architectural components. The architecture and the functionality can then evolve in parallel.

– Alistair Cockburn

risk storming example

schedule milestones

• visualize project milestones
  • Gantt chart
• keep the entire team posted
• pay attention to holidays
  • multiple countries in the case of an international team
• things won’t go as planned, so
  • add safety margin (wiggle room)
  • e.g., an extra week before deadline for fixing bugs

ninety–ninety rule

The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time.

– Tom Cargill, Bell Labs

assign responsibilities and tasks

dependencies

allocate resources

• resource is anything that is available with limits
  • money, time, personnel, equipment, etc.

man-month

– Fred Brooks: The Mythical Man-Month (Brooks, 1974)

estimating time requirement of a task

planning poker

• is a consensus-based, gamified technique for estimation (in agile)
• product owner reads the user story
• members of the group make estimates by playing numbered cards face-down to the table, instead of speaking them aloud
• estimates are then discussed and high and low estimates are explained
  • repeat until estimates converge
• by hiding the figures, the group can avoid the cognitive bias of anchoring
  • where the first number spoken aloud sets a precedent for subsequent estimates

zero: done | tiny: ½ | easy: 1, 2, 3 | medium: 5, 8, 13 | large: 20, 40 | very large: 100 | ∞: huge

optional cards

• ? means unsure
• ☕ means “I need a break”

estimation is guessing

• many developers do not like to estimate
• seemingly simple task can turn out to be difficult
  • some difficulties are hard to foresee
  • bad architectural decision
    • “Architecture is the decisions that you wish you could get right early in a project.” – Ralph Johnson
• make educated guesses instead
  • measure
    • burn down charts, cumulative flow diagram
  • infer from previous tasks

Brooks’s law

Adding manpower to a late software project makes it later.

Assigning more programmers to a project running behind schedule will make it even later. This is because the time required for the new programmers to learn about the project and the increased communication overhead will consume an ever-increasing quantity of available time (Wikipedia contributors, 2024c).

references

wireframe

• a wireframe is an outline / blueprint / concept art of a webpage or application
• can be hand drawn on paper or built out digitally
• provides visual understanding of page structure, layout, user flow, functionality and intended behaviours
• presented to stakeholders before the interface is coded

source: (Bruton, 2022)

wireframing

design prototyping

types of wireframes

low-fidelity wireframe

mid-fidelity wireframe

• provides more precise representations of the layout
• for exploring design ideas, establishing spacing and buttons, and user flow
• still don’t include images, typography or detailed content
  • but show more details regarding components and features
• no colors, grayscale
• usually made with digital tool

source: (Bruton, 2022)

hi-fidelity wireframe

• exploring complex concepts, finalising design
• provides pixel-specific layouts
• usually have actual images and written content
• created using a digital tool
• feature actual typography, detailed features, design elements (logos) and menu systems
• may presented as initial prototypes
  • interactive, clickable

source: (Bruton, 2022)

wireframe map

sitemap

• similar to a wireframe map but for web sites
• for design, documentation
• also for machine processing
  • for web scrawlers
  • sitemap.xml

sitemap as wireframe map

some free tools

• Google Drawings
• draw.io
• Quant-UX
  • open source (self-hosted) or free as a service
• wireframe.cc
  • only public and no export in free plan
• Figma
  • has a limited free plan

references

software design and architecture stack

hierarchy in style guides

• language level:
  • Python: PEP 8 or pep8.org
  • Ruby: Ruby Style Guide
  • Rust The Rust Style Guide
  • etc.
• organization level:
  • Google Style Guides
    • C# at Google Style Guide
    • Google Python Style Guide
    • Google TypeScript Style Guide

not just style guides, also best practices

write idiomatic code

write idiomatic code

for (i = 0; i < 10; i++) {
    console.log(i);
}

[...Array(10).keys()].forEach(i => {
    console.log(i);
});

i = 0
while i < 10:
    print(i)
    i += 1

for i in range(10):
    print(i)

for i in 0..9 do
   puts i
end

(0..9).each do |i|
    puts i
end

(0..9).each {|i| puts i}

clean code

Clean Code: A Handbook of Agile Software Craftsmanship

by Robert C. Martin (2009) (Martin, 2009)

meaningful names

use intention-revealing names

int d; // elapsed time in days

int elapsedTimeInDays;

multi-word names

int elapsedTimeInDays;

public class DataCollector {}

elapsed_time_in_days = 17

avoid disinformation

Do not refer to a grouping of accounts as an accountList unless it’s actually a List (Martin, 2009).

make meaningful distinctions

It is not sufficient to add number series or noise words, even though the compiler is satisfied. If names must be different, then they should also mean something different (Martin, 2009).

def calculate_distance(data: pd.DataFrame) -> pd.Series:
    # do something

def calculate_distance2(data: pd.DataFrame) -> pd.Series:
    # do something else

def calculate_eucledian_distance(data: pd.DataFrame) -> pd.Series:
    # ...

def calculate_levenshtein_distance(data: pd.DataFrame) -> pd.Series:
    # ...

make meaningful distinctions / noise words

Noise words are another meaningless distinction. Imagine that you have a Product class. If you have another called ProductInfo or ProductData, you have made the names different without making them mean anything different (Martin, 2009).

use pronounceable names

If you can’t pronounce it, you can’t discuss it without sounding like an idiot (Martin, 2009).

• Should etid be an integer?
• Should elapsed_time_in_days be an integer?

use searchable names

Single-letter names can ONLY be used as local variables inside short methods. The length of a name should correspond to the size of its scope (Martin, 2009).

it’s OK to do this:

for i in range(10):
    print(i)

it’s NOT OK in a large scope:

int d; // elapsed time in days

names for classes, functions

avoid encodings

with modern IDEs it is pointless to put type or role markers into names

def fnFactorial(iNum):
    if iNum == 1:
        return iNum
    else:
        return iNum * fnFactorial(iNum - 1)

interface IShapeArea // I is also a prefix
{
  void area(); 
}

interface ShapeArea 
{
  void area(); 
}

avoid mental mapping

Readers shouldn’t have to mentally translate your names into other names they already know (Martin, 2009).

don’t pun or use humor

• no inside jokes
• no colloquialisms or slang
• be objective and professional

Say what you mean. Mean what you say (Martin, 2009).

pick one word per concept

it’s confusing to have fetch , retrieve, and get as equivalent methods of different classes (Martin, 2009)

it also helps to search for the term

add meaningful context

Imagine that you have variables named firstName, lastName, street, houseNumber, city, state, and zipcode. Taken together it’s pretty clear that they form an address. But what if you just saw the state variable being used alone in a method? (Martin, 2009)

• adding a prefix?
  • e.g., addrCity, addrStreet, addrState
• as notations are discouraged, use an Address class instead to add context

functions

this section is based on the book Clean Code (chapter 3) by Robert C. Martin (Martin, 2009)

with own examples

functions should be as small as possible

Functions should hardly ever be 20 lines long (Martin, 2009)

• shorter functions are easier to understand

do one thing (single responsibility principle)

import sqlite3
import pandas as pd

con = sqlite3.connect("data.db")
data = pd.read_sql(activity_query, con)

records = []
for woy in range(36, 40):
    for dow in range(1, 8):
        records.append([woy, dow, 0])
empty = pd.DataFrame.from_records(
    records, columns=["week_of_year", "day_of_week", "count"]
)
data = (
    pd.concat([data, empty])
    .drop_duplicates(subset=["week_of_year", "day_of_week"], keep="first")
    .sort_values(["week_of_year", "day_of_week"])
    .reset_index(drop=True)
)
activity = pd.pivot(
    data, index=["week_of_year"], columns=["day_of_week"], values=["count"]
).values
res = con.execute(progress_query)
progress = res.fetchone()[0]

SELECT
    CAST(
        strftime('%W', timestamp) 
        AS INTEGER
    ) AS week_of_year,
    CAST(
        strftime('%u', timestamp)
        AS INTEGER
    ) AS day_of_week,
    count(*) AS count
FROM activity
WHERE
    user_id = 42 AND
    week_of_year > 35 AND
    week_of_year < 40
GROUP BY
    week_of_year,
    day_of_week;

SELECT
    lesson / 50.0 AS progress
FROM activity
WHERE
    user_id = 42 AND
    result = 'success'
ORDER BY lesson DESC
LIMIT 1;

debug tables

the inverse scope law of function names

The longer the scope of a function, the shorter its name should be. Functions that are called locally from a few nearby places should have long descriptive names, and the longest function names should be given to those functions that are called from just one place.

– Robert C. Martin

“longer scope”: more general part of a code

function arguments

def build_empty_dataframe(start, end, cols):
    records = []
    for woy in range(start, end + 1):
        for dow in range(1, 8):
            records.append([woy, dow, 0])
    return pd.DataFrame.from_records(
        records, columns=cols
    )

def query_progress(as_percentage: bool):
    res = con.execute(progress_query)
    progress = res.fetchone()[0]

    if as_percentage:
        return progress * 100
    else:
        return progress

function as interface

DataFrame.to_csv(
    path_or_buf=None, *,
    sep=',',
    na_rep='',
    float_format=None,
    columns=None,
    header=True,
    index=True,
    index_label=None,
    mode='w',
    encoding=None,
    compression='infer',
    quoting=None,
    quotechar='"',
    lineterminator=None,
    chunksize=None,
    date_format=None,
    doublequote=True,
    escapechar=None,
    decimal='.',
    errors='strict',
    storage_options=None
)

no side effects

side effect example

class Something:
    foo = 0
    
    def increase(self, by):
        self.foo += by
    
    def decrease(self, by):
        self.foo -= by
    
something = Something()
print(something.foo)  # 0
something.increase(2)
print(something.foo)  # 2

smth = {"foo": 0}

def increase(what, by):
    return what + by

def decrease(what, by):
    return what - by

print(smth["foo"])  # 0
increase(smth["foo"], 2)  # 2
print(smth["foo"])  # 0
smth["foo"] = increase(smth["foo"], 2)
print(smth["foo"])  # 2

prefer exceptions to returning error codes

• in unix-like systems processes still return 0 if the execution was successful
• but returning error codes in functions are discouraged
• FileNotFoundException is better than ERRCODE_26375
  • meaningful name
  • no mental mapping
  • exception handling syntactically more readable

comments

separating comments

# connect to the database
con = sqlite3.connect("data.db")
# query activity data
data = pd.read_sql(activity_query, con)
# create empty dataframe
records = []
for woy in range(36, 40):
    for dow in range(1, 8):
        records.append([woy, dow, 0])
empty = pd.DataFrame.from_records(records, columns=["week_of_year", "day_of_week", "count"])
# combine empty and sparse dataframe
data = (
    pd.concat([data, empty])
    .drop_duplicates(subset=["week_of_year", "day_of_week"], keep="first")
    .sort_values(["week_of_year", "day_of_week"])
    .reset_index(drop=True)
)
# pivot dataframe
activity = pd.pivot(
    data, index=["week_of_year"], columns=["day_of_week"], values=["count"]
).values

separated functions

def create_empty_dataframe(start_week, end_week):
    records = []
    for woy in range(start_week, end_week+1):
        for dow in range(1, 8):
            records.append([woy, dow, 0])
    return pd.DataFrame.from_records(
        records, columns=["week_of_year", "day_of_week", "count"]
    )

def fill_empty_with_activities(empty, activities):
    return (
        pd.concat([activities, empty])
        .drop_duplicates(subset=["week_of_year", "day_of_week"], keep="first")
        .sort_values(["week_of_year", "day_of_week"])
        .reset_index(drop=True)
    )

def pivot_dataframe(data):
    return pd.pivot(
        data, index=["week_of_year"], columns=["day_of_week"], values=["count"]
    ).values

separated functions - usage

con = sqlite3.connect("data.db")

activities = pd.read_sql(activity_query, con)

empty = create_empty_dataframe(36, 39)

data = fill_empty_with_activities(emty, activities)

activities_matrix = pivot_dataframe(data)

only the comments remained, which can be read as a prose

more bad comments

journal comment

# 2024-10-17 -- Add idiomatic coding examples 
# 2024-10-18 -- Add meaningful names section 

the version tracker keeps better journal

noise comments

# creates an empty dataframe
def create_empty_dataframe(start_week, end_week):
    # ...

don’t write something that is already in the code

closing brace comments

for (i = 0; i < 10; i++) {
    console.log(i);
} // for

modern editors can find (end display) the block endings

Apollo 11 - Colossus 2A

P21VSAVE    DLOAD           # SAVE CURRENT BASE VECTOR
            TAT
        STOVL   P21TIME     # ..TIME
            RATT1
        STOVL   P21BASER    # ..POS B-29 OR B-27
            VATT1
        STORE   P21BASEV    # ..VEL B-7  OR B-5
        ABVAL   SL*
            0,2
        STOVL   P21VEL      # /VEL/ FOR N73 DSP
            RATT
        UNIT    DOT
            VATT        # U(R).(V)
        DDV ASIN        # U(R).U(V)
            P21VEL
        STORE   P21GAM      # SIN-1 U(R).U(V), -90 TO +90
        SXA,2   SET
            P21ORIG     # 0 = EARTH  2 = MOON
            P21FLAG

source, GitHub repository, more about the Apollo Guidance Computer: (Slavin, 2015)

good comments

legal comments

some open source licences should be included to the beginning of the files

informative comments

import re

timestamp = "2024-10-22 09:30:42"
# matches for timestamps in the format of: YYYY-MM-DD HH:MM:SS
re.match(r"\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}", timestamp)

# TODO: this allows invalid month, day, hour, minute and second values
re.match(r"\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}", timestamp)

documentation

def fizzbuzz(i: int) -> str:
    """Fizzbuzz is a game for children to teach them about division.
    It is also a common coding practice.
    
    Parameters
    ----------
    i : int
        Input number tested against division by 3, 5 and 15.
    
    Returns
    -------
    str
        `Fizz` if input divisible by 3, `Buzz` if divisible by 5 and `FizzBuzz` if both.
    """
    result = ""
    if i % 15 == 0:
        result += "FizzBuzz"
    elif i % 3 == 0:
        result += "Fizz"
    elif i % 5 == 0:
        result += "Buzz"
    else:
        result = str(i)
    return result

doctest

def fizzbuzz(i: int) -> str:
    """
    >>> fizzbuzz(3)
    'Fizz'
    >>> fizzbuzz(5)
    'Buzz'
    >>> fizzbuzz(12)
    'Fizz'
    >>> fizzbuzz(15)
    'FizzBuzz'
    >>> fizzbuzz(17)
    '17'
    """
    result = ""
    if i % 15 == 0:
        result += "FizzBuzz"
    elif i % 3 == 0:
        result += "Fizz"
    elif i % 5 == 0:
        result += "Buzz"
    else:
        result = str(i)
    return result

references

code smell

a code smell is a surface indication that usually corresponds to a deeper problem

– Martin Flower (Fowler, 2006)

clean clode violations as code smells

• long method
• long parameter list
• naming
  • notation in names
  • inconsistent names
  • uncommunicative names
• comments
• large class
  • possibly do more than one thing
• a function / class does more than one thing

some code smells

• duplicated code
  • Don’t Repeat Yourself! (a.k.a., DRY principle) (Venners, 2003)
• speculative generality
  • do not generalize the code to solve a potential future problem
  • You aren’t gonna need it (YAGNI)
  • focus on today’s problem
• dead code
  • e.g., a function that is never called
  • editors denote it
  • in Go unused variable is not a warning, but an error
• temporary field
  • “Watch out for objects that contain a lot of optional or unnecessary fields. If you’re passing an object as a parameter to a method, make sure that you’re using all of it and not cherry-picking single fields.” (Atwood, 2006)

conditional complexity

if a and b:
    do_something()

if a or b:
    do_something()

if not (a or (b and not c) and (d or not f)):
    do_something()

conditional complexity

if is_pressure_low() and is_temperature_high():
    do_something()

if is_pressure_low() or is_temperature_high():
    do_something()

if not (
    is_pressure_low()
    or (is_temperature_high() and not is_humidity_low())
    and (is_fall() or not is_raining())
):
    do_something()

class-based smells: alternative classes with different interfaces

If two classes are similar on the inside, but different on the outside, perhaps they can be modified to share a common interface (Atwood, 2006).

class-based smells: data class???

class-based smells: data clumps

If you always see the same data hanging around together, maybe it belongs together. Consider rolling the related data up into a larger class (Atwood, 2006).

class-based smells: refused bequest

If you inherit from a class, but never use any of the inherited functionality, should you really be using inheritance? (Atwood, 2006)

class-based smells: indecent exposure

class-based smells: feature envy

Methods that make extensive use of another class may belong in another class. Consider moving this method to the class it is so envious of.

– Jeff Atwood (Atwood, 2006)

more code smells

this section is based on the book Clean Code (chapter 17) by Robert C. Martin (Martin, 2009)

with own examples

comment related smells

1. obsolete comment

version n-1 (OOP)

# increase class attribute
def increase(self, by):
    self.foo += by

version n (FP)

# increase class attribute
def increase(what, by):
    return what + by

comment related smells

2. redundant comment

# creates an empty dataframe
def create_empty_dataframe(start_week, end_week):

comment related smells

3. commented-out code

def increase(what, by):
    # print(what, by)
    return what + by

not needed, just remove it

class Something:
    foo = 0
    
    def increase(self, by):
        self.foo += by
    
    def decrease(self, by):
        self.foo -= by
    
    # def mutiply(self, by):
    #     self.foo *= by

magic numbers

magic number is an unexplained constant in the code

def calculate_circle_area(r: float) -> float:
    return r * r * 3.141592

PI = 3.141592

def calculate_circle_area(r: float) -> float:
    return r * r * PI

import math


def calculate_circle_area(r: float) -> float:
    return r * r * math.pi

encapsulate boundary conditions

if level + 1 < length:
    do_somthing(foo, bar, level + 1)

next_level = level + 1
if next_level < length:
    do_somthing(foo, bar, next_level)

also increases consistency, the condition needs to be adjusted in one place

denoting blocks

for (i = 0; i < 10; i++) {
    console.log(i);
}

for (i = 0; i < 10; i++)
    console.log(i);

var a = 0;
for (i = 0; i < 10; i++)
    a++;
    console.log(i);

for i in range(10):
    print(i)

a = 0
for i in range(10):
    a += 1
    print(i)

package main
 
import (
    "fmt"
)
 
func main() {
    for i:=0; i<10; i++ {
        fmt.Println(i)
    }
}

fn main() {
    for i in 0..9 {
        println!("{}", i);
    }
}

what could go wrong?

if ((err = ReadyHash(&SSLHashSHA1, &hashCtx)) != 0)
    goto fail;
if ((err = SSLHashSHA1.update(&hashCtx, &clientRandom)) != 0)
    goto fail;
if ((err = SSLHashSHA1.update(&hashCtx, &serverRandom)) != 0)
    goto fail;
if ((err = SSLHashSHA1.update(&hashCtx, &signedParams)) != 0)
    goto fail;
    goto fail;
if ((err = SSLHashSHA1.final(&hashCtx, &hashOut)) != 0)
    goto fail;

fail:
    SSLFreeBuffer(&signedHashes);
    SSLFreeBuffer(&hashCtx);
    return err;

how to measure code quality?

it is hard to objectively measure the quality of code

Halstead metrics

Halstead’s goal was to identify measurable properties of software, and the relations between them (Radon documentation, n.d.).

cyclomatic comlexity

• developed by Thomas J. McCabe in 1976
• quantitative measure of the number of linearly independent paths through the source code
• computed using the control-flow graph of the program

defined as:

$$ M = E - N + 2P $$

cyclomatic comlexity – example

def calculate_progress(
    finished: int,
    total: int,
    as_percentage: bool
) -> float:
    progress = finished / total

    if as_percentage:
        return progress * 100
    else:
        return progress

activity diagram

control flow

cyclomatic comlexity – 2nd example

def calculate_progress(
    finished: int,
    total: int,
    as_percentage: bool,
    foo: bool
) -> float:
    progress = finished / total

    if as_percentage and foo:
        return progress * 100
    else:
        return progress

activity diagram

control flow

Python statements’ effects on cyclomatic complexity

interpretation of cyclomatic complexity – Radon

maintainability index

original (Coleman et al., 1994):
$$ MI = 171 - 5.2 \ln{V} - 0.23G - 16.2\ln{L} $$

Visual Studio derivate:
$$ MI = max\left[0,100 \frac{171 - 5.2 \ln{V} - 0.23G - 16.2\ln{L}}{171}\right] $$

maintainability index – example

def calculate_progress(finished: int, total: int, as_percentage: bool) -> float:
    progress = finished / total

    if as_percentage:
        return progress * 100
    else:
        return progress


def calculate_progress_2(
    finished: int, total: int, as_percentage: bool, foo: bool
) -> float:
    progress = finished / total

    if as_percentage and foo:
        return progress * 100
    else:
        return progress

• maintainability index for a script containing the code above is 63.71
• calculated with Radon

go report

• gofmt: style guide
• go_vet: reports suspicious constructs (Go specific)
• ineffassign: detects ineffectual assignments in Go code
• gocyclo: cyclomatic complexity
• license: checks whether your project has a LICENSE file
• misspell: finds commonly misspelled English words

score	grade
>90
>80
>70
>60
>50
>40
<=40

Go Report Card for Gitea

code chunk permanence in a codebase

WTF per minute

references

V model (Forsberg & Mooz, 1991)

• each phase has output and a review process
  • errors are found at early stage
  • decreases the risk of failure
• testing is done in a hierarchical perspective

requirement analysis review

• can be discussed / reviewed
• even with a customer representative

architecture review

code review

def query_progress(user_id:int) -> float:
    # establish connection
    con= sqlite3.connect("data.db")
    # build query
    progress_query = f"""
    SELECT
        lesson / 50.0 AS progress
    FROM activity
    WHERE
        user_id = {user_id} AND
        result = 'success'
    ORDER BY
        lesson DESC
    LIMIT 1
    ;
    """
    # execute query
    res =con.execute(progress_query)
    progress=res.fetchone()[0]
    return progress

review types by formality

informal

• asking a colleague to have a look at the code
  • they express their opinion and that’s all
  • no documentation
  • no process
• pair programming is also a kind of constant informal review

walkthrough

• not a formal process / review
• led by the author(s)
• the author(s) guide the participants through the work product to achieve a common understanding and to gather feedback
• useful for higher level documents like requirement specification
  • e.g., risk storming, sprint review

technical

• less formal review
• led by the trained moderator or a technical expert
• often performed as a peer review without management participation
• issues are found by experts (e.g., architects, designers)
• technical reviews can vary from quite informal to very formal

(Fagan) inspection (Wikipedia contributors, 2021)

process phases

• planning: the inspection is planned by the moderator

• overview meeting: the author describes the background of the WP

• preparation: each inspector examines the work product to identify possible defects

• inspection meeting: reader reads through the work product, part by part and the inspectors point out the defects

• rework: the author makes changes to the work product according to the action plans from the inspection meeting

• follow-up: the changes are checked to make sure everything is correct

roles

• author: created the work product being inspected

• moderator: the leader of the inspection, who plans and coordinates it

• reader: reads through the documents, while the other inspectors then point out defects

• recorder: documents the defects that are found during the inspection

• inspector: examines the work product to identify possible defects

review types by formality – summary

code review – author’s perspective

you are not your code

code review – reviewer’s perspective

• pay attention to the way you are formulating your feedback
  • phrasing is crucial for your feedback to be accepted
• you and the author are in the same side
• the goal is to deliver higher quality code, not about arguing who was right

use I-messages

talk about the code, not the coder

ask questions

refer to the author’s behavior, not their traits

wrong
You are sloppy when it comes to following the style guide.

Can’t you just configure your IDE properly?

right
I believe that you should pay more attention to the style guide.

Try to enable the auto-formatting in your IDE.

OIR-rule of giving feedback

Observation
Describe your observations in an objective and neutral way. Refer to the behavior if you have to talk about the author. Using an I-message is often useful here.

e.g., “This method has 100 lines.”

Impact
Explain the impact that the observation has on you. Use I-messages.

e.g., “This makes it hard for me to grasp the essential logic of this method.”

Request
Use an I-message to express your wish or proposal.

e.g., “I suggest extracting the low-level-details into subroutines and give them expressive names.”

three filters for feedback

always ask yourself, if your feedback is true, necessary and kind

(from April Wensel (Wensel, 2018) via (Hauer, 2018))

• is it true?
  • avoid statements assuming an absolute truth
  • avoid the words “right”, “wrong”, “never”, “always” and “should”
  • refer to your opinion instead
• is it necessary?
  • does the demanded requested change make the reviewed code better
• it it kind?

praise

• don’t forget to express your appreciation if you have reviewed good code
• praising doesn’t hurt you but will motivate the author
• however, be specific and separate the prasie from the criticism

looks good to me

a review process

• is it understandable (clear)
• is it clean (no code smells)
• does it match the task?
• does it fulfill the task?
  • every DoD point covered?
• is it possible to improve?

how can you tell if the code matches the task?

how to write the commit message?

1. Separate subject from body with a blank line
2. Limit the subject line to 50 characters
3. Capitalize the subject line
4. Do not end the subject line with a period
5. Use the imperative mood in the subject line
6. Wrap the body at 72 characters
   • least important
7. Explain what and why not how
8. Reference the issue!
   • my addition for traceability
   • GH-33 (GitHub), GL-33 (GitLab),
   • or custom identifier for a project (Jira, JetBrains)

review in numbers

#include <stdio.h>
main()
{
    int a,b,c;
    int count = 1;
    for (b=c=10;a="- FIGURE?, UMKC,XYZHello Folks,\
    TFy!QJu ROo TNn(ROo)SLq SLq ULo+\
    UHs UJq TNn*RPn/QPbEWS_JSWQAIJO^\
    NBELPeHBFHT}TnALVlBLOFAkHFOuFETp\
    HCStHAUFAgcEAelclcn^r^r\\tZvYxXy\
    T|S~Pn SPm SOn TNn ULo0ULo#ULo-W\
    Hq!WFs XDt!" [b+++21]; )
    for(; a-- > 64 ; )
    putchar ( ++c=='Z' ? c = c/ 9:33^b&1);
    return 0;
}

#include "stdio.h"
int main (void) {
    int a=10, b=0, c=10;
    char* bits ="TFy!QJu ROo TNn(ROo)SLq SLq ULo+UHs UJq TNn*R\
    Pn/QPbEWS_JSWQAIJO^NBELPeHBFHT}TnALVlBLOFAkHFOuFETpHCStHAU\
    FAgcEAelclcn^r^r\\tZvYxXyT|S~Pn SPm SOn TNn ULo0ULo#ULo-WH\
    q!WFs XDt!";
    a = bits[b];
    while (a != 0) {
        a = bits[b];
        b++;
        while (a > 64) {
            a--;
            if (++c == 'Z') {
                c /= 9;
                putchar(c);
            } else {
                putchar(33 ^ (b & 0x01));
            }
        }
    }
    return 0;
}

references

V model (Forsberg & Mooz, 1991)

• each phase has output and a review process
  • errors are found at early stage
  • decreases the risk of failure
• testing is done in a hierarchical perspective
• review is a testing process usually without executing the code

test pyramid

what is a unit test?

def fizzbuzz(i: int) -> str:
    """
    >>> fizzbuzz(3)
    'Fizz'
    >>> fizzbuzz(5)
    'Buzz'
    >>> fizzbuzz(15)
    'FizzBuzz'
    >>> fizzbuzz(17)
    '17'
    """
    result = ""
    if i % 15 == 0:
        result += "FizzBuzz"
    elif i % 3 == 0:
        result += "Fizz"
    elif i % 5 == 0:
        result += "Buzz"
    else:
        result = str(i)
    return result

what really is a unit?

• defined as a single behaviour exhibited by the system under test
  • usually corresponding to a requirement
• it may imply that it is a function or a module / method or a class
  • depending on the paradigm
• functions / methods, modules or classes don’t always correspond to units
• “only entry points to externally-visible system behaviours define units”
  • by Kent Beck (Beck, 2002)

unit vs integration testing

unit test example

def fizzbuzz(i: int) -> str:
    result = ""
    if i % 15 == 0:
        result += "FizzBuzz"
    elif i % 3 == 0:
        result += "Fizz"
    elif i % 5 == 0:
        result += "Buzz"
    else:
        result = str(i)
    return result

from fizzbuzz import fizzbuzz


def test_fizzbuzz():
    assert fizzbuzz(3) == "Fizz"
    assert fizzbuzz(5) == "Buzz"
    assert fizzbuzz(10) == "Buzz"
    assert fizzbuzz(12) == "Fizz"
    assert fizzbuzz(15) == "FizzBuzz"
    assert fizzbuzz(17) == "17"

arrange, act, assert pattern

parts of a unit test

def test_fizzbuzz():
    # arrange
    test_input = 3
    # act
    result = fizzbuzz(test_input)
    # assert
    assert result == "Fizz"

arrange, act, assert(, annihilate) pattern

parts of a unit test

arrange
set up the testing environment

(e.g., create objects)

act
call the tested unit

assert
compare the result of the ‘act’ step to the expected value

annihilate
free resources

automatic in modern languages

how to unit test this funciton?

def query_progress(user_id: int) -> float:
    # establish database connection
    con = sqlite3.connect("data.db")
    # build query
    progress_query = f"""
    SELECT
        lesson / 50.0 AS progress
    FROM activity
    WHERE
        user_id = {user_id} AND
        result = 'success'
    ORDER BY
        lesson DESC
    LIMIT 1
    ;
    """
    # execute query
    res = con.execute(progress_query)
    progress = res.fetchone()[0]
    return progress

separate business logic from persistence

architectural styles provides patterns to separate the business logic from the persistence layer

unit testing usually targets the business logic

which was embedded into the query in the previous example

SELECT
    lesson / 50.0 AS progress
FROM activity
WHERE
    user_id = 42 AND
    result = 'success'
ORDER BY lesson DESC
LIMIT 1;

separated business logic

def query_last_finished_lesson(
    user_id: int
) -> float:
    # establish database connection
    con = sqlite3.connect("data.db")
    # build query
    query = f"""
    SELECT lesson
    FROM activity
    WHERE
        user_id = {user_id} AND
        result = 'success'
    ORDER BY lesson DESC
    LIMIT 1;
    """
    # execute query
    res = con.execute(query)
    return res.fetchone()[0]

def calculate_progress(
    finished: int, total: int
) -> float:
    return finished / total


def calculate_user_progress(
    user_id: int, total: int
) -> float:
    f = query_last_finished_lesson(user_id)
    return calculate_progress(f, total)

separated data connection

def query_last_finished_lesson(
    con: sqlite3.Connection,
    user_id: int
) -> float:
    # build query
    query = f"""
    SELECT lesson
    FROM activity
    WHERE
        user_id = {user_id} AND
        result = 'success'
    ORDER BY lesson DESC
    LIMIT 1;
    """
    # execute query
    res = con.execute(query)
    return res.fetchone()[0]

def establish_database_connection(
    path: str = "data.db"
) -> sqlite3.Connection:
    return sqlite3.connect(path)

mocking

• the whole unit test suite should be able to run in milliseconds
  • to give immediate feedback
• slow elements of the software should be mocked
  • e.g., database, network connection
• part of arrange step

test doubles – mock object types

there is no open standard for categories

• dummy
• stub
• spy
• mock
• fake

test doubles – test dummy

require 'sinatra'

get '/user-statistics' do
  return {}.to_json
end

test doubles – test stub

require 'sinatra'

get '/user-statistics' do
  data = {}
  data['name'] = 'Marvin'
  data['id'] = 42
  data['registration'] = '2019-10-02'
  data['progress'] = 0.84
  data['activity'] = [
    [2, 0, 2, 3, 5, 3, 2],
    [5, 2, 4, 4, 0, 3, 4],
    [6, 3, 0, 6, 8, 3, 0],
    [9, 7, 4, 7, 0, 9, 9]
  ]
  return data.to_json
end

test doubles – test spy

test doubles – test fake

require 'sinatra'

def generate_progress
  rand.round(2)
end

def generate_activity_matrix
  result = []
  (1..4).each do |_w|
    daily = []
    (1..7).each {|_d| daily.push rand(10)}
    result.push daily
  end
  result
end

get '/user-statistics' do
  data = {}
  data['name'] = 'Marvin'
  data['id'] = 42
  data['registration'] = '2019-10-02'
  data['progress'] = generate_progress
  data['activity'] = generate_activity_matrix
  return data.to_json
end

test doubles – test mock

A mock is dynamically created by a mock library (the others are typically produced by a test developer using code). The test developer never sees the actual code implementing the interface or base class, but can configure the mock to provide return values, expect particular members to be invoked, and so on. Depending on its configuration, a mock can behave like a dummy, a stub, or a spy.

– Mark Seemann (Seemann, 2007)

test-driven development (TDD)

• write test before writing the tested code
• without the called unit the test will fail
  • the called function does not exist
• write code, that makes the test pass
• improve the code quality
  • e.g., make it clear and clean
  • both the test and tested code

red

• test only one thing at a time
• the test should be very simple
• increase the complexity of the test cases continuously
• mock the (external) dependencies
  • bit later

green

• use the possible simplest code to pass the test
• it does not matter if the solution is ‘ugly’
  • but the test must pass
• as soon as the test passes, this step is done
  • and all of the old tests as well

refactor

Refactoring is a disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior.

– Martin Fowler (Fowler, n.d.)

• on code level
  • style guide, best practices, idiomatic code
• on architecture level
  • design patterns like SOLID, DRY, etc.
• part of day-to-day programming
  • ‘campground rule’: leave the code better than you found it

test-driven development – fizzbuzz example

def fizzbuzz():
    pass

def fizzbuzz(i):
    pass

def fizzbuzz(i):
    return "Fizz"

def fizzbuzz(i):
    return "Fizz"

def fizzbuzz(i):
    if i % 3 == 0:
        return "Fizz"
    elif i % 5 == 0:
        return "Buzz"

def fizzbuzz(i):
    if i % 3 == 0:
        return "Fizz"
    elif i % 5 == 0:
        return "Buzz"

def fizzbuzz(i):
    if i % 15 == 0:
        return "FizzBuzz"
    elif i % 3 == 0:
        return "Fizz"
    elif i % 5 == 0:
        return "Buzz"

def fizzbuzz(i):
    if i % 15 == 0:
        return "FizzBuzz"
    elif i % 3 == 0:
        return "Fizz"
    elif i % 5 == 0:
        return "Buzz"

def fizzbuzz(i):
    if i % 15 == 0:
        return "FizzBuzz"
    elif i % 3 == 0:
        return "Fizz"
    elif i % 5 == 0:
        return "Buzz"
    else:
        return str(i)

from fizzbuzz import *

def test_fizzbuzz():
    assert fizzbuzz(3) == "Fizz"

from fizzbuzz import *

def test_fizzbuzz():
    assert fizzbuzz(3) == "Fizz"
    assert fizzbuzz(5) == "Buzz"

from fizzbuzz import *

def test_fizzbuzz():
    assert fizzbuzz(3) == "Fizz"
    assert fizzbuzz(5) == "Buzz"

from fizzbuzz import *

def test_fizzbuzz():
    assert fizzbuzz(3) == "Fizz"
    assert fizzbuzz(5) == "Buzz"
    assert fizzbuzz(15) == "FizzBuzz"

from fizzbuzz import *

def test_fizzbuzz():
    assert fizzbuzz(3) == "Fizz"
    assert fizzbuzz(5) == "Buzz"
    assert fizzbuzz(15) == "FizzBuzz"

from fizzbuzz import *

def test_fizzbuzz():
    assert fizzbuzz(3) == "Fizz"
    assert fizzbuzz(5) == "Buzz"
    assert fizzbuzz(15) == "FizzBuzz"
    assert fizzbuzz(17) == "17"

from fizzbuzz import *

def test_fizzbuzz():
    assert fizzbuzz(3) == "Fizz"
    assert fizzbuzz(5) == "Buzz"
    assert fizzbuzz(15) == "FizzBuzz"
    assert fizzbuzz(17) == "17"

As the tests get more specific, the code gets more generic.

– Robert C. Martin, The Cycles of TDD (Martin, 2014)

transformation priority premise

1. ({} -> nil) from no code at all to code that employs nil
2. (nil -> constant)
3. (constant -> constant+) a simple constant to a more complex constant
4. (constant -> scalar) replacing a constant with a variable or an argument
5. (statement -> statements) adding more unconditional statements
6. (unconditional -> if) splitting the execution path
7. (scalar -> array)
8. (array -> container)
9. (statement -> tail-recursion)
10. (if -> while)
11. (expression -> function) replacing an expression with a function or algorithm
12. (variable -> assignment) replacing the value of a variable

coding kata

too strict TDD

>>> "Nov 08, 13:11"[3:5]
' 0'
>>> "Nov 08, 13:11"[4:6]
'08'

def extract_day(s: str) -> int:
    return int(s[4:6])

def test_extract_day():
    actual = extract_day("Nov 08, 13:11")
    expected = 8
    assert actual == expected

behaviour-driven development (BDD)

Title (one line describing the story)

Narrative:
As a [role]
I want [feature]
So that [benefit]

Acceptance Criteria: (presented as Scenarios)

Scenario 1: Title
Given [context]
  And [some more context]...
When  [event]
Then  [outcome]
  And [another outcome]...

Scenario 2: ...

acceptance criteria

Title (one line describing the story)

Narrative:
As a [role]
I want [feature]
So that [benefit]

Acceptance Criteria: (presented as Scenarios)

Scenario 1: Title
Given [context]
  And [some more context]...
When  [event]
Then  [outcome]
  And [another outcome]...

Scenario 2: ...

acceptance test-driven development

• extends TDD and BDD
• instead of a unit, ATDD focuses on the acceptance criteria of the whole system
• advocates writing acceptance tests before developers begin coding

test format like BDD, example from (Wikipedia contributors, 2022):

Given Book that has not been checked out
And User who is registered on the system
When User checks out a book
Then Book is marked as checked out

readme driven development

test coverage

• the percentage of the code lines ‘protected’ or covered by tests

def fizzbuzz(i: int) -> str:
    result = ""
    if i % 15 == 0:
        result += "FizzBuzz"
    elif i % 3 == 0:
        result += "Fizz"
    elif i % 5 == 0:
        result += "Buzz"
    else:
        result = str(i)
    return result

def fizzbuzz(i: int) -> str:
    result = ""
    if i % 15 == 0:
        result += "FizzBuzz"
    elif i % 3 == 0:
        result += "Fizz"
    elif i % 5 == 0:
        result += "Buzz"
    else:
        result = str(i)
    return result

def fizzbuzz(i: int) -> str:
    result = ""
    if i % 15 == 0:
        result += "FizzBuzz"
    elif i % 3 == 0:
        result += "Fizz"
    elif i % 5 == 0:
        result += "Buzz"
    else:
        result = str(i)
    return result

from fizzbuzz import fizzbuzz


def test_fizzbuzz():
    assert fizzbuzz(15) == "FizzBuzz"
    assert fizzbuzz(3) == "Fizz"

from fizzbuzz import fizzbuzz


def test_fizzbuzz():
    assert fizzbuzz(15) == "FizzBuzz"
    assert fizzbuzz(3) == "Fizz"
    assert fizzbuzz(5) == "Buzz"

from fizzbuzz import fizzbuzz


def test_fizzbuzz():
    assert fizzbuzz(15) == "FizzBuzz"
    assert fizzbuzz(3) == "Fizz"
    assert fizzbuzz(5) == "Buzz"
    assert fizzbuzz(17) == "17"

how to measure code quality?

it is hard to objectively measure the quality of code

when unit tests are not more than a measure

• zombie scrum: doing something without heart, without its essence
• if you write unit tests just to increase the test coverage they loose its function
  • and collect badges:

what to test?

def calculate_progress(
    finished: int,
    total: int,
    as_percentage: bool,
) -> float:
    progress = finished / total

    if as_percentage:
        return progress * 100
    else:
        return progress

from progress import calculate_progress


def test_progress():
    total = 50
    for i in range(total + 1):
        expected = i / total
        actual = calculate_progress(i, total, False)
        assert actual == expected


def test_progress_percentage():
    total = 50
    for i in range(total + 1):
        expected = i / total * 100
        actual = calculate_progress(i, total, True)
        assert actual == expected

test the edge cases!

def calculate_progress(
    finished: int,
    total: int,
    as_percentage: bool,
) -> float:
    progress = finished / total

    if as_percentage:
        return progress * 100
    else:
        return progress

how to find edge cases

• interval boundaries
• requirements
• defining of done
• acceptance criteria of BDD-style scenarios
  • extended user user stories

Story: Account Holder withdraws cash

As an Account Holder
I want to withdraw cash from an ATM
So that I can get money when the bank is closed

an acceptance criterion:

Scenario 1: Account has sufficient funds
Given the account balance is $100
 And the card is valid
 And the machine contains enough money
When the Account Holder requests $20
Then the ATM should dispense $20
 And the account balance should be $80
 And the card should be returned

a test function:

def test_withdraw():
    account = Account(balance=100)
    withdraw_money(account, 20)
    assert account.balance == 80
    account = Account(balance=10)
    withdraw_money(account, 20)
    assert account.balance == 10

legacy code

what is legacy code?

Code without tests is bad code. It doesn’t matter how well written it is; it doesn’t matter how pretty or object-oriented or well-encapsulated it is. With tests, we can change the behavior of our code quickly and verifiably. Without them, we really don’t know if our code is getting better or worse.

– Michael Feathers, Working Effectively with Legacy Code: Preface (Feathers, 2004)

• there is a change request, which results on code change
• the test suite is like a safety net that can prevent that a code change breaks an existing function

the footprint, the compass and the flag figures by Lorc under CC BY 3.0 via game-icons.net

the legacy code dilemma

When we change code, we should have tests in place. To put tests in place, we often have to change code.

– Michael Feathers, Working Effectively with Legacy Code (Feathers, 2004)

(Part I / Chapter 2)

the legacy code change algorithm

1. identify change points
2. find test points
3. break dependencies
4. write tests
5. make changes and refactor

sensing

We break dependencies to sense when we can’t access values our code computes.

– Michael Feathers, Working Effectively with Legacy Code (Feathers, 2004)

e.g., misspelled function name

separation

We break dependencies to separate when we can’t even get a piece of code into a test harness to run.

– Michael Feathers, Working Effectively with Legacy Code (Feathers, 2004)

seams

A seam is a place where you can alter behavior in your program without editing in that place.

– Michael Feathers, Working Effectively with Legacy Code: Part I / chp. 4 (Feathers, 2004)

A seam is a place in the code that you can insert a modification in behavior. […] One way to take advantage of a seam is to insert some sort of fake.

– tallseth via Stackoverflow | CC BY-SA 3.0

• using inheritance
  • subclass can do the same as parent class
  • but can be extended with sensing code
• preprocessing seam
• link seam
  • using build script, e.g., “same” class in different directory

changing the software

testing approaches

black box

• examining / testing the functionality without knowing the inner structure
• works at all levels: unit, integration, system, acceptance
• also for debugging a legacy code

white box

• testing the internal structure as opposed to its functionality
• often associated to unit testing, but also works on higher levels (i.e., integration, system)

smoke testing

• preliminary testing
• smoke tests are a subset of test cases that cover the most important functionality of a component or system
• set of tests run on each new build to verify that it is testable before sent to the test team

source: Smoke testing (software), Wikipedia (Wikipedia contributors, 2024a)

rubber duck debugging

references

what to automatize?

in a software development context

linting

auto formatting

• there are automatic code formatters for more and more languages
  • that can reformat the source code to align with the style guide
• usually triggered by saving the file
• usually configurable to align with organization / project specific rules
• some examples:
  • Python: ruff (among others)
  • Ruby: rubocop
  • Go: gofmt

well configured editor

version control systems

file sharing issue

lock-modify-unlock solution

copy-modify-merge solution

copy-modify-merge solution #2

centralized vs. distributed version control system

example: subversion

example: git, mercurial

feature branching

branching strategies

• branching is more than just separating workspace, work-in-progress code from released
  • also for managing stable (released) versions
  • and bugfixing though multiple versions

continuous integration (CI)

• emerged from extreme programming
• considered an agile approach
• gives immediate feedback
  • the integration (merging) will fail if two branches are not compatible
  • and build the integrated software
• also gives opportunity to do testing on the built software…

continuous integration

continuous integration environment

build script

example: build script of the course website

scheduled build

nightly build

continuous deployment

continuous deployment environment

• extension of a continuous integration environment
• deployment is another stage in the build script
• same triggers as in a CI environment (not just the scheduler)

blue–green deployment (Wikipedia contributors, 2024a)

shadow deployment

• two servers are maintained (“live” and “shadow”)
• for testing the performance and stability requirements
  • on success, the release can be deployed to the live server as well
• specialized strategy, complex and (relatively) expensive to set up

canary deployment

• deployment in an incremental fashion
• starts with a small number of users
• and continues until 100% is reached
• allows to test updates in live environment
  • on small groups of users
  • before deploying to many users
  • may involve telemetry

devops

• software development + IT operations
  • collaboration
• agile mindset, set of principles (GitLab, 2022)
  • automation of the SDLC
  • collaboration and communication
  • continuous improvement
  • focus on user needs with short feedback loops
• relies on automatization, CI and CD
• to build, test and release better software
  • frequently, reliably, rapidly

what tools to use?

• CI and CD became a fundamental part of software development
  • got integrated into services like GitHub, GitLab, BitBucket, JetBrains Space
• some solutions:
  • Jenkins
    • open source, self hosted
  • GitHub Actions
    • integrated to code hosting, free options
    • example: course repository
  • GitLab Pipelines
    • integrated to code hosting, free options
  • Travis CI
    • free for open source projects
  • Circle CI
    • free options

automatized review

automatized review – example

• CI services integrated into the code hosting / developer platforms
• code changes can ba annotated by automatized review findings
  • usually at a pull request level
• a bit slower feedback than running static analysis or test locally

vulnerability alerts

[tool.poetry.dependencies]
python = "^3.12"
numpy = "^1.26.3"
pandas = "^2.2"
geopandas = "^1.0"
networkx = "^3.2.1"
osmnx = "^1.6.0"
matplotlib = "^3.8.2"
seaborn = "^0.13.0"
contextily = "^1.3.0"
opencv-python = "^4.9.0"
pyaml = "^23.9.7"
pyogrio = "^0.7"
pyarrow = "^15.0.0"
scipy = "^1.12.0"
haversine = "^2.8.1"
mapclassify = "^2.6.1"
openpyxl = "^3.1.2"
ecomplexity = "^0.5.2"
structlog = "^24.1.0"
h3 = "^3.7.7"
pandarallel = "^1.6.5"
jinja2 = "^3.1.4"
tabulate = "^0.9.0"

dependabot example

remote: Resolving deltas: 100% (5/5), completed with 4 local objects.
remote: 
remote: GitHub found 1 vulnerability on pintergreg/software-engineering's default branch (1 high). To find out more, visit:
remote:      https://github.com/pintergreg/software-engineering/security/dependabot/1

dependencies

• choose carefully the software packages / components your software will depend on
• use well maintained software modules
• unmaintained modules have potential vulnerabilities
• aim for loose coupling regarding the dependency
  • makes it easier to replace if needed

daily work

• select a task from backlog
• read and understand it
• create a feature branch

• write code, possibly using TDD
• local unit testing, checks
• open a pull request (initiate merge)

• pull requests used to manage review
• trigger for CI to run automated tests, static code analysis, generate reports, etc.
• while your work is reviewed, start working on another task / review other’s work

• if everything is fine, the task is done
• eventually the change will be in production at the end users
• as a part of a release
  • a release can group together multiple changes
  • then, scrum start a new iteration, or kanban continues as always
• using some kind of deployment strategy

release versioning

semantic versioning

calendar versioning

format examples:

• YYYY.MINOR.PATCH
  • micro is used instead of patch
• YYYY.MM.MINOR.PATCH

“CalVer is a versioning convention based on your project’s release calendar, instead of arbitrary numbers.” | calver.org

ZeroVer: 0-based versioning

“Your software’s major version should never exceed the first and most important number in computing: zero.” | 0ver.org

• e.g.: 0.4.1
• popular among open source software projects
  • some may reach 1.0.0 eventually

Fibonacci releases

KDE / Plasma 6.1 series release schedule

interruption

interruption

• the greatest “enemy” of a developer is interruption
• the code is one thing, the logic behind it is another
  • takes time to understand
• context switching is costly
  • switching between tasks
• that is why it is advisable to define small tasks during the sprint planning
  • 1–4 hours, but ideally closer to 1
  • preventing interruption
  • a programmer probably get one uninterrupted 2-hour session in a day (Parnin, 2013)

the cost of interruption

• according to a study, the average lost time per major interruption is 23 minutes (Mark et al., 2008)
  • for developers, it could be worse
  • according to another study it is at least 15 minutes (Parnin, 2013)
• “getting back to the exact state of mind you were at right before an interruption is nearly impossible” (To, 2018)
• interruptions can be planned and unplanned

planned and unplanned interruptions

techniques to minimize context switching