This the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

tangly llc Blog

A collection of articles about our journey through agile product development, digitalization, and software craftsmanship.

2021

Effective Working in Digital Age

Effective Working in Digital Age?

2021 02 03 head

Home office is the new way of working. A lot of friends and business partners are working from home. Our meetings are mostly online. We share documents and results through digital platforms.

I truly hope we continue to work online in the future. I would like to mix home office, local coworking space, and regular office attendance. I am convinced less commute time and a greener approach to work are certainly great improvements.

I would also state the obvious. My infrastructure at home is better. I have a better Internet connection. My display is a high-end 4K screen instead of a cheap HD version. The coffee is also better.

My life would be more productive and fun if more people better understand how to work in a digital world.

Here a few advices to improve our experience.

Schedule an Appointment

  • Send me a calendar invitation in the ics format. Never email me asking for an appointment, please provide directly the invitation. Do not send invitations in Outlook proprietary format, I use macOS and Linux, not Microsoft and Office.
  • Send me a meaningful invitation. Provide an agenda, attendee list and attach documents I shall read before the meeting. Add the link and password for the conference such as Zoom, Google Hangouts, or Microsoft Team.

Test your connection before initiating your call. Mute your microphone to avoid disturbing the call attendees. Please buy a professional headset.

Use tools such as Trello and Miro to have more collegial meetings.

Write a Meeting Protocol

If you are responsible for a meeting please:

  • Provide an invitation with attendee list, agenda, and as attachment the documents I should read if I want to attend.
  • Write a protocol describing the activities of the meeting. A well-written protocol has the following sections
    • A copy of the agenda you sent to all attendees
    • A short description of the discussion
    • A list of decisions
    • A list of tasks to implement. Each task has a description, a due date and a responsible.

Please send the link to the protocol for review to all attendees. Define the deadline when the protocol will be released.

Any meeting without such a protocol was in fact just a gathering to eat donuts and drink coffee. Modern organizations have coffee break for such gatherings.

Share a Document

The rules to share a document are:

  • Avoid sending me a document per email. Send me the link to the document in a shared repository I have access to. I mean everybody is using Google Drive, DropBox, or Microsoft One Drive.

The rules to edit a document are:

  • Do not assume I am using Microsoft Office. I use and strongly prefer LibreOffice. I support open source software and format. Microsoft Office formal definition is not even implemented in Office.
  • When modifying a shared document, consider using the review mode as being polite to your team members.
  • PDF is the de facto standard to exchange documents in the wide world. Acrobat Reader is free to use on all major platforms. Acrobat Reader supports a review mode.

Communicate

  • Please use a chat application like Slack to communicate. Do not email me and call 20 minutes later to ask me if I read your email.
  • Please call me to discuss a complicated topic or schedule a telco Do not start an email trail to identify potential solutions to something complicated

Work on a Project

  • Use Git and set all the code artifacts under source code management.
  • Work with text formats. Avoid Office or LibreOffice formats for your documents You cannot compute differences on such documents.
  • Consider AsciiDoc as your preferred format. AsciiDoc is semantically equivalent to DocBook. You can write whole books with AsciiDoc.

Lessons Learnt

Adapt your analog processes to the digital world. Do not just copy them. Improve your processes and remove spurious activities.

Enjoy your online interactions. Go regularly drink a coffee or a beer with your peers. We are flesh and blood entities.

Why Is Java Path Class Better?

Why Is Java Path API Class Better?

2021 02 02 head

File System NIO

Java NIO New Input Output is an alternative IO API for Java, meaning alternative to the standard Java IO and Java Networking API’s. Java NIO offers a different IO programming model than the traditional IO APIs.

One major feature of interest is the support of alternate file systems. An in-memory file system is ideal to test file related operations in unit and integration tests.

Unit Tests

The Jimfs library implements an in-memory file system. You can find other implementations of in-memory file systems if you want to use another library.

The approach is ideal to write unit tests working with files. You cannot assume to have access to a well-defined file system in a generic continuous integration pipeline. Unit tests using a in-memory file system can be deployed in a continuous integration pipeline.

When you look at the implementation of Path.of or Paths.get, you will see:

public static Path of(String first, String... more) {
    return FileSystems.getDefault().getPath(first, more);
}

So, while this method and similar ones are very convenient, using them will imply you want to access your default file system. The one your JVM is running on, not your in-memory file system.

Hence, when wanting to make sure your code works against in-memory file systems, you must make sure to never call these helpers methods. Instead, you should always use the FileSystem or a Path instance as an anchor.

Depending on your project and your libraries, this is quite a challenge to pull off [1].

Paths

Alternative file system operations require all methods should use path objects instead of file objects. File objects contain the name of the accessed file but loose all references to the underlying file system. All operations using file objects implicitly use the default file system.

Path objects encapsulate the file and the file system. Operations are aware of the underlying file system and behave as expected.

FileSystem fs;  1

Files.createDirectory(fs.getPath(ORGANIZATION, "crm/"));  2
Files.copy("myFile", fs.getPath(ORGANIZATION, "crm/")), StandardCopyOption.REPLACE_EXISTING);
  1. Define a file system
  2. Work with file system to create paths and avoid using helper methods using the default file system

Consider the Java java.io.File class as a legacy abstraction. Avoid using it and move over to the Java path abstraction.

Remember the java.nio.Path class has a reference to a file system, the java.io.File class only use the default file system.

The javadoc definition states that a file is

  • An abstract representation of file and directory path names.
  • User interfaces and operating systems use system-dependent pathname strings to name files and directories. The javadoc definition states that * The java.nio.file package defines interfaces and classes for the Java virtual machine to access files, file attributes, and file systems.
  • This API may be used to overcome many of the limitations of the java.io.File class.

Use file instances to manipulate abstract file path names. Use path instances to manipulate files.

Beware that

File file = new File("/tmp/myfile.txt");

Path path = file.toPath();

// is implemented as

FileSystems.getDefault().getPath(this.getPath());

If you need to convert a file instance to a path instance hosed in the desired file system please write

File file = new File("/tmp/myfile.txt");

Path path = fileSytem.getPath(file.getPath());

Learnt Lessons

Legacy libraries heavily use File objects in their algorithms and their public interface. Such libraries do not work with in-memory file systems.

The simplest approach is to always provide an API returning streams in addition of file systems methods. It is always possible to map a stream to a file system specific streams for read and write operations.

The more complex approach is to replace all File objects with Path objects. Ideally the Java community should declare the File class as a deprecated one. Another solution would be to extend File with an internal path instance. This approach could break backward compatibility.


  1. I regularly open issues in various open source libraries to request an API extension to support Path. Often the maintainer of the library is not aware you need Path to unit test with in-memory file systems.

LeSS Lessons Learnt

LeSS Lessons Learnt

2021 02 01 head

Scaling Agile, Lean and Scrum to the whole product development department is hard work.

You start a change journey in murky waters. Best to communicate a lot, strengthen your teams and have total support of upper management.

Technical agile coaching usefully describes the work I do. Agile coaching helps your business to become more successful, often improve the way you plan and deliver software.

Technical agile coaching focuses on how people design products, write code and deliver high quality products.

My aim is the same as other agile coaches. I improve the agility of the organizations I work with. I use coaching, mentoring, teaching and facilitation techniques just as other coaches do.

My main focus is how the source code is written, the design defined, and the architecture formulated.

Our goal is to deliver a highly successful product.

Agile Approach

My own interpretation of agile as doctrine is:

  • Reduce the distance between problems and problem-solvers.
  • Practice Gemba.
  • Take small steps, collect data, validate every step, update your approach. Apply the Plan-Do-Check-Act PDCA loop. Clean up as you go.
  • Lead and cycle-time are measures of the organizational agility. Track them.

Do not ask for permission when you want to continuously improve. A professional is supposed to improve. This is a core responsibility of your teams and team members.

You have the retrospectives to identify and plan your improvements. You must allocate time for implementing the improvement in every sprint. Scrum is clear, at least one improvement measure is a high priority backlog item in the next sprint [1].

Ways of working

The following three levels of coaching are commonly needed:

Organizational LeSS Coaching

A coach works with multiple teams and the management to improve the organization and its structure. A clear vision of how a LeSS organization could be:

2021 02 01 coaching
  • Organizational structure and design,
  • LeSS,
  • Role of management,
  • Adoption and goals setting,
  • Managing an organizational improvement backlog,
  • Changing management practices from controlling day-to-day work to building capability,
  • Managing work: product backlogs and how the organization manages them,
  • Dealing with interruptions,
  • Human Resource practices such as appraisals and career paths,
  • Facilitating organizational release/roadmap planning and retrospectives.

Team LeSS Coaching

A coach works with at most a few teams to improve their team-working and LeSS practices. It is common for a coach to take on the Scrum Master role. A team coach typically focuses on:

  • Team responsibilities with self-managing teams,
  • Facilitation,
  • Improving the team’s decision-making and conflict resolution,
  • Transparency in the team,
  • Making organizational impediments visible,
  • Improving the relationship between the Development Team and the Product Owner,
  • Product Ownership (of both the team, the PO, and other stakeholders),
  • Role and contribution of the team’s management,
  • Improve Scrum practices (and technical practices),
  • Educate and coach the team’s (future) Scrum Master.

Technical Practices Coaching

A coach works with (or on) a team on their actual codebase in order to improve the technical practices and adopt agile development techniques.

A technical coach is an expert in software development techniques

2021 02 01 technical agility

LeSS strongly emphasizes technical agility and promotes associated good practices. High-quality products requires well-trained professional developers and mastery.

Examples are simple design, refactoring, unit testing, test-driven development and acceptance test-driven development.

A coach typically focuses on:

  • Discovering “code/design smells”,
  • Places where code/design could be improved,
  • Explaining modern, “clean” code that is simple and easier to change and maintain,
  • Refactoring “smells” into clean code,
  • Writing unit tests,
  • Test-driven development,
  • Test Automation Continuous integration and continuous delivery,
  • Specification by Example (Acceptance Test-Driven Development),
  • Efficient and effective working practices (IDE, automation),
  • Applying design patterns.

In all our mandates one major activity is improving legacy code.

Legacy code is Code without tests

Legacy code is Profitable code that we feel afraid to change

How Much Coaching?

The most successful LeSS adoptions we have seen had the following structure:

One internal and one external coach

This pair provides the overview of the LeSS adoption. They both coach management but are also involved with team and technical coaching.

External team coaches who help the teams become better and focus on training the Scrum Masters.

External technical coaches who focus on training internal technical coaches

Have some technical coaches work with the teams, but let them focus on training internal coaches. After that reduce (not eliminate!) the external technical coaching or let the external coach move to a new area.

Be Patient, The Time Horizon is Years

  • Trust your people,
  • Create opportunities,
  • Establish a learning culture,
  • Thrive for craftsmanship,
  • Let the team use internal social pressure,
  • Eliminate specialization, push mastery,
  • At the beginning follow the LeSS rules,
  • Extended responsibility, rounded products provides a room and freedom for better decisions.

Remember

LeSS is Scrum, it is Large Scaled Scrum Feature teams are economical. Cross-functional Technical excellence is the essence of a quality solution One product owner for the whole product provides focus.

The product owner is where the money is. Depending on your organization put him where the budget comes from.

Move from a component owner to become a component mentor. He is responsible to teach others how to adapt and evolve the component.

References


  1. Sadly the Scrum Guide revision of 2020 removed this rule. They delite this PDCA mechanism.

Software Development for Students

Software Development for Students

2021 01 04 head

Computer science students shall achieve technical excellence when reaching bachelor level. They should be proficient with industry good practices. Students understand the theory and principles behind these practices. Experience with one of the best of breed tools enforces learning and automatism.

The focus of a technical university is to teach the concepts and methods. The tools are used to practice and to increase student learning.

Source Code Version Management with Git

Git is the standard tool for distributed source code management system. It is supported by all major development IDE and continuous integration environments.

First year students shall master the basic git commands to write their semester projects.

Second year students shall master the git commands used to work in teams. Pull requests are a standard approach to review team colleague works and consolidate new functions into the application.

Modern Java Software Development

Java is the preferred programming language for the majority of teaching institutions around the world. The programming language has evolved the last ten years. Students should learn the current way of writing code in Java.

The actual Java approach is to use streams and renounce in most cases to use instructions such as for or while loops or conditional if statements. Delegation and lambda constructs often replace inheritance design.

Since JDK 16 Java provides Algebraic Data Types with records and sealed classes. Records implement Product Types. Sealed classes implement Tagged Unions also called sum types.

The IntelliJ IDEA IDE is now the industrial standard for the Java and often Python world. The IDEA environment promotes the use of modern Java constructs. Currently, constructs such as records, sealed classes, closeable resources, switch instructions as expressions are new approaches to write legible Java code. Concepts such as Optional, Streams, immutable objects, and unchecked expressions promote more functional programming styles.

Gradle is our preferred build tool for Java projects. The approach is straight forward.

Technical Excellence

The DevOps and software craftsmanship movement strongly promote technical excellence and intrinsic quality in software development.

720

Students shall be fluent with

  • Static code analysis. Intellij IDEA provides an analyze function for static code analysis. The IDE is free for students.
  • Unit tests and integration tests Use the seminal libraries JUnit5, Mockito, and AssertJ
  • Continuous integration pipeline Use the CI pipeline of your product management tool; either GitLab, GitHub, Bitbucket, or Azure DevOps.

Documentation

You should only check text artifacts into a git repository. Modern approaches with Markdown and Asciidoc syntax promote this axiom.

PlantUML is a text based diagramming approach. All major syntax and in particular all UML diagram types are supported. Asciidoc has native support for diagrams in general and PlantUMl in particular. Asciidoc supports additional diagrams and diagramming tools such as Mermaid.

Agile and Scrum

Agile and Scrum are the standard methods to develop software products in the industry. All students shall excel in agile development approaches. Scrum is the most used approach and should be taught to all computer science students.

Students shall understand the agile concepts and main constructs. When working in a Scrum environment, they have used the backlog, issues, stories, definition of done. They have planed their projects with a vision, a roadmap, epics, and sprints.

Gitlab as a Student’s Tool

2021 01 04 gitlab

Gitlab is an integrated solution for software development in teams and DevOps approaches. It is the official tool provided in our Computer Science Department.

Students learn computer science principles and hands-on how to develop software applications. They will use their learnings to develop commercial software components when working after the completion of their studies.

Gitlab as a Lecturer’s Tool

All the source code and artifacts of a student class are available on Gitlab. The development history over a project or a semester are visible in Git. The lecturers can access the artifacts or use scripts to extract the data necessary to set the grades for the students. The grading process can be standardized. The decision process is part of the gitlab data and is auditable.

The student series is a set of blogs showing how to use central tools for modern software development.

Gitlab for Bachelor Students

Gitlab for Bachelor Students

720

Computer science students shall be proficient with a basic set of software development techniques.

  • Technical excellence is a mandatory skill for all software developers.
  • Agile development approaches are state of the industry and used in almost all software development initiatives.
  • DevOps approaches are used in modern organizations. They predate agile development approaches.

Technical schools should provide in-depth experience in these areas to prepare their students to their future activities after completing their bachelor.

The below mind map consolidates concepts, tools and approaches based on the GitLab tooling established in some universities. The approach emphasizes agile and DevOps concepts and approaches.

students proficiencies

Italic terms identify concrete tools. Green font marks Gitlab features. Light blue background concepts map to technical university guidelines.

You should develop software applications using agile and DevOps techniques. You shall put your source code under a version management system. The actual industrial standard is git, a distributed version control system - DVCS -.

More sophisticated source code management systems based on git with additional agile and DevOps functionalities are:

  • Github is a closed source solution and most well-known solution. Microsoft acquired them in 2020. Some Swiss technical universities - ZHAW, University of Bern - use the academic github campus program. CI and CD pipelines are not part of the Github product and must be installed separately. The academic program means you receive github licenses. Your organization is responsible to run the software on premises.
  • Gitlab is an open source solution and strong contender to github. CI and CD pipelines are part of the product.
  • Bitbucket is a closed source solution of Atlassian. CI and CD pipelines are part of the product. Atlassian also owns JIRA, Confluence and Trello.
  • Azure Devops is a closed source solution and Microsoft answer for .NET development. CI and CD pipelines access is under definition by Microsoft. They slowly open their product to additional development stacks such as Java, Python.
  • Gitea is a community managed lightweight code hosting solution written in Go. The project is hosed on Github. It is published under the MIT license.
  • Gogs is a painless self-hosted Git service. The project is hosed on Github.
  • Gitolite allows you to set up git hosting on a central server. It has fine-grained access control and many more powerful features.

School often select GitLab due to his open source license and the tremendous DevOps functionality. The gitlab solution supports source code versioning, DevOps pipelines, agile development approaches, and quality insurance for the produced artifacts.

Basic Git Knowledge for Students

Studying the history of software development practices shows the raise and fall of SCCS, CVS, ClearCase, Visual Source Safe, or Subversion. The current most popular and used tool is Git.

Any software developer or web designer can create a git hosted project with a few commands in the shell.

Git has 3 different areas for your code:

Working directory
The area that you will be doing all of your work in (creating, editing, deleting, and organizing files)
Staging area
The area where you will list the changes that you have made to the working directory
Repository
Where Git permanently stores the changes you have made as different versions of the project
git init 1
git clone https://gitlab.enterpriselab.ch/tangly/tangly-os.git 2

git add README.md 3
git add * 4

git commit -m "commit message, should be clear and legible" 5
git push 6
git pull 7

git status 8
git remote 9
git --version 10
  1. Create a new repository for the root folder of your project, the folder .git is created and contains all the git tracking information.
  2. Clone the repository tangly-os and copy all files into the current folder.
  3. Add the README file to git.
  4. Add all files to git.
  5. Commit files.
  6. Push files to the remote repository you cloned in step 2. Your changes are now public and available to other developers.
  7. Pull files from the remote repository you cloned in step 2. Your local repository contains the last public changes of other developers.
  8. Returns status information about your local project.
  9. Returns the list of remote repositories your project is connected to. Often you have just one remote to synchronize with.
  10. Displays the version of your git installation.

See our blog Git Local Repositories For the Impatient.

See our blog Git Branches for the Impatient for more advanced commands.

CI Pipelines for Students

720

The minimal gradle build file of a Java project is very simple and compact

apply plugin: 'java'

Multi-module Gradle build file requires an additional gradle.settings file identifying your modules.

include 'net.tangly.bdd'
include 'net.tangly.bus'
include 'net.tangly.commons'

The initial build file for each module is the same as for a minimal gradle project. You can now compile, test and check your application

gradle build 1
gradle test 2
gradle check 3
gradle clean 4
  1. Compile the simple or multi-module Java project.
  2. Execute the unit tests of the project.
  3. Perform static analysis checks if you define them.
  4. Clean the output folders and delete all generated files.

You are now ready to create a complete continuous integration pipeline of your project in gitlab. A complete .gitlab-ci.yml configuration for a gradle project would be:

image: openjdk:15

variables:

before_script:
  - export GRADLE_USER_HOME=`pwd`/.grade

stages:
  - build
  - test

build:
  stage: build
  script: ./gradlew --build-cache assemble
  cache:
    key: "$CI_COMMIT_REF_NAME"
    policy: push
    paths:
      - build
      - .gradle

test:
  stage: test
  script: ./gradlew check
  cache:
    key: "$CI_COMMIT_REF_NAME"
    policy: pull
    paths:
      - build
      - .gradle

Java Development Environment

Warning:

Please prefer a workstation powered with Linux or macOS if you are developing in Java, Groovy or Python.

You will be more efficient and avoid all the problems related to virus scanners, missing packages or arcane path length problems.

  1. Install the latest LTS JDK and latest JDK on your workstation.
  2. Install IntelliJ IDEA IDE. Students can request a student license and work with the Ultimate edition. The community edition is free for everybody.
  3. Use the static analysis function of IntelliJ IDEA has a static analysis feature.
    1. Select the file or package to inspect in the project view.
    2. Click the right mouse button to display the context menu and select the menu item Analyze/Inspect Code…​. The findings will teach you how to write better Java code and eliminate common errors.
  4. Install Gradle to build your application

IntelliJ IDEA environment provides all the tools to work efficiently locally. All the tools identified in the above mindmap are supported in IDE and associated plugins. Students can work offline and learn the tools in a gentle manner.

The students can use the IDEA Static Code Analyzer without any additional configuration. The quality is good enough and the feature empowers junior developers to learn modern and correct Java code.

More advanced scenarios use SpotBugs and the associated IDEA plugin.

How to Write Unit Tests for a Java Application

You should write unit and integration tests for your Java application using

How to Document Your Project

Static code generators are becoming the standard approach for technical documentation. One huge advantage is you can check the whole documentation into your git repository. You have now a synchronisation between a released version of your application and the corresponding documentation. Both are stored in the same repository and can be marked with a version label You can use

IntelliJ has native support for markdown and an excellent plugin for Asciidoc.

Gitlab provides the Pages concept to create and publish a complete documentation of a product.

Diagrams

720

PlantUML is a simple and powerful tool for diagramming. The tool is compatible with git philosophy and static website generators.

The IntelliJ Asciidoc is used to write technical documentation and to display embedded PlantUML diagrams. Asciidoc format has native support for PlantUML diagrams.

Students have training in technical notations such as UML, BPNM, C4 and others. They can now apply these notations to their projects using plantUML.

Agile and Scrum

Students have training in agile development and the Scrum framework. Gitlab allows them to apply these Agile Delivery Concepts to their semester and bachelor projects. Lecturers have access to the agile and Scrum artifacts and can qualify the quality of the work.

Advanced Git for Complex Team Development

Once you master the basic commands, consider learning advanced commands like:

Merge

Merge your branch my-branch back into trunk

git commit –m "Some commit message" 1
git checkout trunk 2
$ git merge my-branch 3
  1. …​develop some code and commit it in the currently selected my-branch
  2. switched to branch trunk
  3. merge the branch my-branch into trunk

Rebase

Rebase trunk into your branch my-branch. Note that after a rebase, the commit ids of the rebased branch are new. A rebase is not simply a moving of commits around in the history. Git actually creates brand-new commits with new commit ids and permanently deletes the old commits.

You should prefer to rebase only with local branches. Avoid rebasing shared branches. If you rebase a branch that was pulled from GitHub or GitLab, and then push it back, the server will reject it.

git rebase my-branch trunk

Squash

Git squash is a technique that helps you to take a series of commits and condense it to a few commits.

So let’s see how to squash the last three commits to a single commit.

git rebase -i HEAD~3

The following content will be shown:

pick 474bfdc2 3rd last commit
pick 1bffc15c 2nd last commit
pick 474bf0c2 last commit

For the commits that you want to squash, replace the pick instruction with fixup, so it becomes:

fixup 474bfdc2 3rd last commit
fixup 1bffc15c 2nd last commit
pick 474bf0c2 last commit

Cherry Picking

Cherry-picking in Git means to choose a commit from one branch and apply it onto another. This is in contrast with other ways such as merge and rebase which normally apply many commits onto another branch.

You can display the hashes of your commits in trunk with the command

git log --oneline trunk

Now you can cherry-pick the commit you are interested in and apply it into your branch.

git checkout my-branch
git cherry-pick <commit-hash>

Local and Remote Branch

Read the post Git Branches for the Impatient.

Pull Request

Read the documentation of GitLab, BitBucket or GitHub.

References

Meaningful Identifiers

Meaningful Identifiers

2021 01 02 head

Identifiers are a key concept to successfully model complex business domains. They permeate how our modern civilisation functions. Could you imagine a country without car registration number, internet addresses, postal addresses or social security numbers?

Identifiers are identifiers.

Identifiers cannot be changed at any time. Updating identifiers means introducing mapping relations between the old and the new values.

In computer science, identifiers IDs are lexical tokens that name entities.
Identifiers are used extensively in virtually all information processing systems.
Identifying entities makes it possible to refer to them, which is essential for any kind of symbolic processing.

External Well-Defined Identifiers

The best identifiers are externally well-defined identifiers. A unique and well-defined authority is responsible to define and publish them. Ideally the authority should only create new identifiers and never delete existing ones. The published information is available to all interested parties.

Examples are

  • Currency names as defined in ISO standard 4217,
  • Country abbreviations as define in ISO standard 3166,
  • Social security numbers are well documented in the country they are used,
  • Car numbers are well-known and documented.

Try to use global externally defined identifiers as much as possible. Always document the authority in charge to create these identifiers.

Examples you are using are

  • IBAN account numbers are defined worldwide through ISO 13616,
  • Bank connections are defined as Business Identifier Codes through BIC ISO 9362. The registration authority is SWIFT,
  • Car numbers and country acronyms are defined worldwide,
  • Passport numbers are defined worldwide. Each country has its own structure and conventions,
  • Social security numbers are a counter-example. There are currently only defined locally.

External Managed Identifiers

You shall document the source of truth for all externally defined and managed identifiers.

The source of truth shall work as a benevolent responsible. It should support all interested parties and guaranty their identifiers can be trusted.

Internal Managed Identifiers

One possible approach is to define a naming convention such as

Year - Acronym - Counter

An example of such an identifier would be

2021-CNT-0001 (Contract number 1 in year 2021)
2020-EVT-0101 (Event number 101 in year 2020)

Variations are

  • A full date as prefix, our system uses this convention to name invoices,
  • A more readable acronym, our ERP system uses the word invoice to name invoices.

Beware if you export internal managed identifiers you become a source of truth. You are responsible to document these identifiers, and the process to create new ones. You shall also guaranty that existing identifiers are never recycled. If your identifiers are strings, you shall clearly define the supported languages, and character sets.

Lessons Learnt

Always try to find an external well-defined identifier for your entities. Beware of legal aspects when you are using government defined identifiers such as social security numbers, passport numbers or car numbers.

Avoid by all means to use an internal privately defined identifier as a public one. You will bear the burden to document, manage and evolve the scheme as soon as you do this.

Related concepts are discussed in our blog series

Value Objects as Embedded Entities

Value Objects as Embedded Entities

2021 01 01 head

This post discusses the value object pattern, and the factory pattern which are tactical patterns in the domain driven design DDD approach.

Value objects represent typed values, that have no conceptual identity in your domain. They can hel you write better code that is less error-prone, more performant and more expressive.

Value objects define the second kind of domain objects besides entities. Their main characteristic is immutability:

Attributes of a value object never change.

A second characteristic is they do not have external identifiers. Unlike an entity two value objects with the exact same properties are considered equal.

A third one is they should be self-validating. A value object shall verify the validity of its attributes when being created. If any of its attributes are invalid, the object should not be created and an error or exception should be raised.

Value Objects do have attributes and methods as entities. Attributes of value objects are immutable therefore methods of value objects can only be queries without side effects. Operations never command a change to the internal state of a value object. We can pass value objects to clients without worrying they change them.

Many objects can be modeled as value objects instead of entities because they are defined through their attributes. These objects measure, quantify or describe things in the domain model. Because they are so easy to handle we should model domain abstractions as value objects as often as possible.

Addresses, Phone Numbers, Email Addresses

Some value objects are part of any commercial application domain. Postal addresses, phone numbers and email addresses are canonical value objects. Other candidates are bank account IBAN numbers, job titles, job descriptions, company legal statuses.

This means that these objects do not have an object identifier and should not be stored in a separate table if stored in a SQL database. Value objects shall be embedded objects and be part of an object such as a person.

A person is an entity. It should have an external identifier and has a postal address, multiple phone numbers, and one or more email addresses.

Check your domain model and validate these assumptions.

  • Postal address, phone numbers, email addresses are value objects and have no external or internal identifiers.
  • They are always owned by other value objects or more often by entities such as persons, companies, delivery forms.
  • They are immutable objects.
  • Business processes can replace such an object with a new instance without impeding other domain instances.
  • Programmed in modern Java these value objects should be records to guaranty immutability at the language level. As a bonus you get the hashCode and equals methods for free.

Invoice Lines

Invoice lines are lines containing details in an invoice. Invoice lines are always value objects belonging to exactly one invoice.

To complicate matter, invoice lines have different types such as a regular line with a specific article, quantity, unit price and computed price. Often VAT aspects must be considered and computed for the overall invoice. An invoice line can also be a subtotal for a set of regular invoice lines.

Warning:

VAT definition and computation is often a murky domain. The VAT value can be dependant of the article, the company selling it and the client buying it. Often when public administration defines a domain and associated business rules the world becomes quite complicated and ambiguous. The VAT administrative laws in Switzerland are more than 2000 pages of illegible text.

The complete definition of this business domain is not part of this article.

The following design challenges exist

  • Serialization libraries such as JSON Jackson library cannot handle invoice lines with multiple Java class types. A workable approach is described and implemented with the open source component Gleam. The solution is to define selectors to instantiate the correct Java class.
  • Invoices are complex objects and ideal candidates for a document oriented persistent approach. Invoices should not be stored in a relation based persistence store, meaning a SQL database. To store them in a table you have to model the various type of lines and add a technical identifier to each invoice line instance.

Factories

The factory pattern in Domain Driven Design DDD can be seen as a super pattern for the Gang of Four (GoF) creational patterns. Factories are concerned with creating new entities and value objects. They also validate the invariants for the newly created objects. We can place a factory on the entity or value object itself or on an independent object.

Factories that are declared on the same object they create are either factory methods or prototype methods. The factory method creates a completely new object from the method parameters. The prototype method uses an existing instance to derive a new object.

When the creation logic is complex or has dependencies that are not needed by the created object it is best to create a separate factory. This factory could provide multiple ways to create new instances.

Value Objects in Persistent Store

We encourage experimenting with the MicroStream approach for small projects. The effort to persist a Java object graph is very small.

You can always move to a no SQL solution when your application is successful and time is come to harden it. Another standard but cumbersome approach is to move to JPA.

Related concepts are discussed in our blog series

2020

Support Comments for Static Hugo Website

Support Comments for Static Hugo Website

2020 12 02 head

There are times in software when you are facing a problem, you find an off-the-shelf solution, and a few lines of code later it just works.

Today I lived one of those moments. The whole configuration of comments and modification of the Docsy theme took around one hour.

Motivation

I wanted to add a comments section to this blog for a while. I was reluctant to implement one myself the classical way, using an external service such as disqus. This blog is a static website created with Hugo and does not use a database. All pages are written with Asciidoc, stored in a Git repository and rendered using Hugo. I wanted to store comments in a personally owned repository and protect my readers from advertisement and personal information collection.

Meeting Utteranc.es

Some information on the Internet about utteranc.es is wrong. Facts are:

  • Utterances uses a github repository to store comments associated with your website articles and pages.
  • Your website can be hosted anywhere. You do not need to host your static website repository on github; but you can.
  • Readers wanting to comment must login once with their github account to authorize the utteranc.es application. Anonymous commenting is not supported. This restriction is reasonable for a technical website which readers are technical affine and already have a github account.
  • Utterances is free. It claims to have no tracking, no advertisement, and no lock-in. The comments are stored in your repository as GitHub issues.
  • Utterances layout looks nice and can be styled.

Utterances is an open source app built and maintained by Jeremy Danyow. He announced the project in 2018. It provides a lightweight web widget <iframe> to add a comment section to a website.

When someone posts a comment, the Utterances bot literally leaves a corresponding comment on the issue. In other words, each page gets its own issue with a comment feed, and so you benefit from the entire feature set of issues out-of-the-box.

  • Comments are stored as issues inside a GitHub repository of your choosing.
  • You can edit, report and delete comments. These functions are great for moderation.
  • If you are watching the repository, you will get email notifications when comments are posted to your site.

The other wonderful aspect about Utterances is how simple the entire setup really is.

Setting Up Utterances

The process is described in details on utteranc.es. The steps are:

  1. Create a public GitHub repository.
  2. Install the utteranc.es app on the repo you would like to host comments on. You can used the repository of your blog, or setup a dedicated repository. This option means your blog or website does not need to be hosted on GitHub.
  3. Configure the code snippet on the utteranc.es website and copy it.
<script
  async
  src="https://utteranc.es/client.js"
  repo="<OWNER>/<NAME>" 1
  issue-term="title"
  theme="github-light"
  crossorigin="anonymous"
</script>
  1. <OWNER>/<NAME> is the name of your repository. In my case it would be marcelbaumann/tangly-team-comments.

Configure Docsy Theme

I use the Hugo theme Docsy to style the Open Source Components. I put some effort to have a configurable extension through site prameters.

Create the comments partial for utterances

The partial file contains the code to display for commenting using utterances. It is a modified version of the snippet you can generate on the utteranc.es website.

Create a partial in docsy/layout/partial. I named my partial comments-utterances.html and add the following snippet.

<div class="page-blank">
    <div id="utterancers_thread"></div>
    <script src="https://utteranc.es/client.js"
            repo= {{ .Site.Params.comments_utterances_repo }}                   1
            issue-term= {{ .Site.Params.comments_utterances_issue_term }}       2
            theme= {{ .Site.Params.comments_utterances_theme }}                 3
            crossorigin="anonymous"
            async>
    </script>
</div>
  1. Github repository where the issues are stored
  2. See utteranc.es documentation how the reference to a page, or an article can be tailored
  3. See utteranc.es documentation about supported themes

Edit the blog content file

The template docsy/layouts/blog/content.html is edited to include the above defined partial file. The original file supports connecting to disqus. I replaced the disqus specific code with the inclusion of the partial defined above for utterances comment support. Perhaps Docsy will introduce a more generic mechanism in the future to integrate commenting systems. I assume they are waiting on Hugo to first provide such a mechanism. Currently, Hugo and Docsy only provide out of the box support for Disqus.

<div class="td-content">
	<h1>{{ .Title }}</h1>
	{{ with .Params.description }}<div class="lead">{{ . | markdownify }}</div>{{ end }}
	<div class="td-byline mb-4">
		{{ with .Params.author }}{{ T "post_byline_by" }} <b>{{ . | markdownify }}</b> |{{ end}}
		<time datetime="{{  $.Date.Format "2006-01-02" }}" class="text-muted">{{ $.Date.Format $.Site.Params.time_format_blog  }}</time>
	</div>
	{{ if (and (not .Params.hide_readingtime) (.Site.Params.ui.readingtime.enable)) }}
	    {{ partial "reading-time.html" . }}
	{{ end }}
	{{ .Content }}
	{{ if (.Site.Params.comments_utterances) }}
		<br />
		{{ partial "comments-utterances.html" . }}
		<br />
	{{ end }}

	{{ partial "pager.html" . }}
</div>

Add four site variables to config.toml

The above changes try to be generic and are configured through site variables in the site configuration file under the [params] block. Therefore you do not need to edit partial files to tailor them.

# flag indicating if the utterances (https://utteranc.es/) should be displayed
comments_utterances = true

# GitHub repository name where all comments are stored. The repository can be the same as the website repository or a different one.
comments_utterances_repo = "marcelbaumann/tangly-team-comments"

# Encoding used to map a site page to a GitHub issue. See utterances (https://utteranc.es/) documentation.
comments_utterances_issue_term = "pathname"

# Theme used by utterance on the website. See utterances (https://utteranc.es/) documentation.
comments_utterances_theme = "github-light"

Advocate Zero Bug Policy in Your Projects!

Advocate Zero Bug Policy in Your Projects!

2020 12 01 head

At many customer sites I visit, I see a lot of effort being put into classifying and managing bugs.

Their bug list may contain more than a thousand items. All are neatly documented with screenshots, ways to reproduce the problem and a severity and impact classification.

A change management board meets every few weeks and decide which bugs have now a higher priority and should urgently be fixed. It means an angry customer called someone at C level and now his request must be handled. Some of these bugs were discovered years ago. The discussion can take hours and involve up to ten people. Clearly, many expensive hours have been invested to properly administer these bugs.

Creating a list of errors provides an organized inventory of technical debt. Spending time on evaluating and administering bugs with the goal of tracking them is just wasteful. Please stop doing such activities.

The solution is elegant and simple: Implement the Zero Bug Policy.

Stop Managing Bugs, Start Focusing on Quality!

How to transition?

Zero-Bug policy does not mean bug-free code in production; it means striving to eradicate all known bugs.

2020 12 01 fix it or delete it

The rigorous Zero-baseline approach will force us to clean our code.

Let us categorize the bugs you have and deal with them accordingly.

  1. You have a few critical bugs that crash the system or don’t have a workaround. Prioritize and fix those. Your customers will be very thankful. Fix at least a handful of these critical problems in each sprint.
  2. A big set of bugs that are already in production. They have been in your bug tracking system for quite a while. Delete those. If it is a costly problem your customers will fill a report. Treat such bugs as critical bugs.
  3. Establish a software craftsmanship culture. Stop delivering products with known errors. Correct them before delivery. Your product owner should reject any feature with known errors.

Bugs have a tendency to become obsolete pretty quickly in applications that are updated frequently. The steps to reproduce them become irrelevant, the functionality changes, the impact of the bug often gets lower.

Please, don’t waste your time trying to update the documentation related to these bugs. You may lose a few valid ones, but trust me, someone will report it again if it’s still happening.

Clean Code and Happy Customers

Bugs are like water leaks.

If you leave them for too long, they can spoil your merchandise and slow down your business.

Finally, to ensure this system works.

  • The bugs that the team decides to fix must be prioritized over regular work and must get fixed now.
  • In the beginning there will be more bugs to work through, as some bugs that were deleted will be reported again.
  • As the team shifts its focus to improved quality there will be fewer of them. This is truly a binary system with no wiggle room.
  • Bugs are either fixed now or deleted, there is no severity and no haggling about the classification of a bug! Good practice is before fixing the glitch write a unit test to reproduce the problem. You guarantee the users will never see again the same error. It is called non-regression testing and make your customers very happy.
  • Are you ready to shift your focus from managing bugs to ensuring quality? And if not, why should I buy your product?

References

See the excellent articles from Yassal Sundman working for the crisp company.

The Quest for an Agile Company: Five Years tangly llc

The Quest for an Agile Company: Five Years tangly llc

2020 11 02 head

Agile Motivation

An agile company is a business with the ability to quickly adapt to market changes [1]. Agile organizations must be fast-moving and flexible, have rapid response to unexpected obstacles, lead change, be in touch with customers. They have a continuous competitive advantage, and a flat organizational structure for quick communication [2].

In other words, an agile organization is optimized for a VUCA - Volatile, Uncertain, Complex, Ambiguous - environment. The organization should provide vision, understanding, courage and adaptability.

There is simply no way to build tomorrow’s essential organizational capabilities - resilience, innovation and employee engagement - atop the scaffolding of 20th century management principles.

Gary Hamel, 2007

Agile Culture

The LeSS movement states the main approach clearly:

Culture follows Structure.

Craig Larman

The successful steps are always:

Change the structure
One approach is value streams organization and product development with cross-functional teams advocated in LeSS.
Change the processes
Work using agile approaches such as Scrum, Kanban, eXtreme Programming. Study the advices of the LeSS guys.
Culture will follow
The organization must nurture the change process and eliminate barriers. The servant-leader approach is ideal.
2020 11 02 waisfisz levels of culture

To Waisfisz, an organization’s culture is defined by how its parts relate to each other. Moreover, it refutes the the way we do things here model by reinforcing that your internal culture is defined by its differences from other organizations. This model lends from group dynamics where groups are not only defined by their internal identity but also by their differences to other groups.

Clearly recognize not changing the structure and just renaming roles and processes will not trigger any change. See the Larman laws:

  1. Organizations are implicitly optimized to avoid changing the status quo middle- and first-level manager and specialist positions and power structures.
  2. As a corollary to (1), any change initiative will be reduced to redefining or overloading the new terminology to mean basically the same as status quo.
  3. As a corollary to (1), any change initiative will be derided as purist, theoretical, revolutionary, religion, and needing pragmatic customization for local concerns — which deflects from addressing weaknesses and manager/specialist status quo.
  4. As a corollary to (1), if after changing the change some managers and single-specialists are still displaced, they become coaches/trainers for the change, frequently reinforcing (2) and (3).
  5. Culture follows structure.
Craig Larman

In the language of social psychology [3] we can state:

It turns out that if we engage in a behavior, and particularly one we had not expected that we would have, our thoughts and feelings toward that behavior are likely to change.

This might not seem intuitive, but it represents another example of how the principles of social psychology, in this case the principle of attitude consistency lead us to make predictions that would not otherwise be that obvious.

Principles of Social Psychology, 1st Internation Edition

Agile Anti-Patterns

You have to acknowledge you must change the structure of your company. The current approach is to remove functional silos and reorganize along value streams. A stream is in charge of s set of related products and directly communicates with the users of these products.

Attempting to change an organization’s culture is a folly, it always fails. Peoples’ behavior the culture is a product of the system; when you change the system, peoples’ behavior changes."

John Seddon
  • The major anti-pattern is the refusal to change structure of your organization. Without changing structure you will fail.
  • Your organization is unique but not esoteric. The actual approaches will work. You need to tailor them.
  • Agile is around for a quarter of a century. Accept that if you changing now, you are late. You will find thousands of experience reports.
  • Accept it will take time. The Agile Fluency Model provides a potential roadmap.
  • Agile transformation works only if your organization thrives to satisfy customer’s needs and achieve technical excellence.

Please compare your change process against these anti-patterns. Correct if needed.

Lessons Learnt

2020 11 02 agile digital architecture

Agile is now mainstream. More than 80% of all software development initiatives claim to be agile. Digital and agile enterprise are two sides of the same coin. Almost all research and development departments are implementing agile processes to achieve faster and better results. The organizations moving now to agile approaches are laggards.

Beware that laggards will fight against any change. I still hear upper management statements such as:

  • I do not like and do not want to hear the word agile
  • We are a different industry. Agile cannot work in our environment.
  • We have our processes, we cannot change them. And, agile is anyway chaos.

Find a champion and get rid of the most vocal laggards. If no champion can be found either move to another company or renounce agile approaches. Be realistic, you will fail to have any lasting success in such an environment.

Embrace reality. At least eighty percent of all software and research activities claim to follow agile approaches. This war is won. We are moping the last islands of resistance.

For me, it is similar to the introduction of object-oriented approaches in analysis, design, and realisation of software projects. In the late nineties of last century the war was won. It took more than ten years to take care of the last luddites.

Our company tangly llc works steadily and continuously to advance agile and lean values. Our structure and processes actively support agile values. We are very happy with the current results.

Literature

  • [1] Reinventing Organizations: A Guide to Creating Organizations Inspired by the Next Stage of Human Consciousness, Frédéric Laloux, 2014
  • [2] The Future of Management, Gerry Hamel, Harvard Business School Press, 2007
  • [3] An Organizational Cultural Perspective, Waisfisz Bob, 2015, Hofstede Culture Center Strategy, ITIM International

Reference Codes

Reference Codes

2020 11 01 head

Reference codes are an old concept. They are often used in relational database schema definition as enumeration values. The database specialists often call them lookup tables.

Reference codes are dynamic enumeration values. By dynamic, we mean you can add values without recompiling the source code.

In modern DevOps environments where deployment of a new version is performed in minutes this advantage is somewhat shallow. We view reference codes more as strongly typed and performant tags.

How can you easily add the concept of reference codes in your application? We provide an initial setup approach requiring mere minutes to add codes to your application.

Enumerations as Codes

As a professional designer you have declared sets of fixed values as enumeration in your source code. Now you can extend your enumeration classes with the code interface.

Additional information are available under link:../../../docs/bus/businessmodels/ Below the source code in modern Java is

public interface Code {
    int id();
    String code();
    boolean isEnabled();
}

public enum EnumCode implements Code {
    CODE_TEST_1, CODE_TEST_2, CODE_TEST_3, CODE_TEST_4, CODE_TEST_5(false);

    private final boolean enabled;

    EnumCode() { this(true); }

    EnumCode(boolean enabled) { this.enabled = enabled; }

    @Override
    public int id() { return this.ordinal(); }

    @Override
    public String code() { return this.toString(); }

    @Override
    public boolean isEnabled() { return this.enabled; }
}

The above approach is a low impact transformation of your application. Once your enumeration types implements the code interface you are free to migrate selected types to external persistent and extensible solutions.

Persistent Codes

Codes can be defined and stored as JSON sets or in SQL table.

The Java code for the JSON approach is

/**
 * Utility method to read all code values from a JSON file using the org.json library.
 *
 * @param clazz   class of the reference code
 * @param factory placeholder to pass the class constructor as lambda expression
 * @param path    path to the JSON file containing the code values
 * @param <T>     class of the reference code
 * @return code type and all its values
 * @throws IOException if a file access error occurred
 */
public static <T extends Code> CodeType<T> build(Class<T> clazz, CodeFactory<T> factory, Path path) throws IOException {
    try (Reader reader = Files.newBufferedReader(path, StandardCharsets.UTF_8)) {
        Iterator<Object> iter = new JSONArray(new JSONTokener(reader)).iterator();
        List<T> codes = new ArrayList<>();
        while (iter.hasNext()) {
            JSONObject value = (JSONObject) iter.next();
            T code = factory.create(value.getInt("id"), value.getString("code"),
            value.getBoolean("enabled"));
            codes.add(code);
        }
        return CodeType.of(clazz, codes);
    }
}

The JSON files containing the code values can be stored as resource files and delivered either with application or in a separate jar file.

The Java code for the SQL table is quite similar:

/**
* Utility method to read all code values from a relational database table using Java regular API.
*
* @param clazz class of the reference code
* @param factory placeholder to pass the class constructor as lambda expression
* @param dataSource data source to the database to read from
* @param tableName name of the table containing the code values
* @param <T> class of the reference code
* @return code type and all its values
* @throws SQLException if a database access error occurred */
public static <T extends Code> CodeType<T> build(Class<T> clazz, CodeFactory<T> factory, DataSource dataSource, String tableName) throws SQLException {
    final String SQL_QUERY = "SELECT id, code, enabled FROM " + tableName;
    try (Connection connection = dataSource.getConnection(); Statement statement = connection.createStatement();
        ResultSet resultSet = statement.executeQuery(SQL_QUERY)) {
        List<T> codes = new ArrayList<>();
        while (resultSet.next()) {
            T code = factory.create(resultSet.getInt("id"), resultSet.getString("code"), resultSet.getBoolean("enabled")); codes.add(code);
        } return CodeType.of(clazz, codes);
    }
}

At the database level you can define integrity rules in the schema to increase data quality.

Design

You have the freedom to model set of values as enumeration, reference codes or tags. Each approach has specific advantages and tradeoffs. The costs of moving from one approach to another are quite small. Just choose one solution, implement it and track its adequacy and user acceptance.

We provide a Java library BUS implementing these constructs. More information is available under Open Source Components.

The unit tests contain examples for enumeration codes, JSON codes and SQL database stored codes.

Related concepts are discussed in our blog series

Creating a technical Website with Hugo and Asciidoc

Creating a technical Website with Hugo and Asciidoc

2020 10 01 head

I hosted my technical blogs on blogger for years. If I correctly remember I started to use blogger ten years ago. It is a great tool with some drawbacks. Over the years the drawbacks sting more and more:

  • Their editor is brittle, with limited functionality and unreliable. The generated HTML is not legible and does not support concepts such a micro fragments, meta information or simple functions such as source code highlighting.
  • The last quarters they started to tinker with their editor and output format. My older posts are now a mess and can only be open in HTML mode. If I switch to their WYSIWYG editor the layout is destroyed and random fonts family and sizes are displayed. Worse the blogs are also displayed mangled to the readers even if I do not edit them. This destruction of all older blogs and the missing migration path were killer criteria.
  • Blogger does not support modern markup language such Markdown or Asciidoc. Blogger uses a proprietary and not easy to port format.

It is time to find an alternative, and I have to confess I am a huge fan of Asciidoc.

OK, so why not go with Markdown? Don’t get me wrong, there is nothing bad with Markdown - except that no one should probably use it when Asciidoc and Asciidoctor are available. I’m writing all my documents with Asciidoc.

The following needs are identified:

  1. I shall write blogs with the selected solution. The blogs shall be written in Asciidoc. The blog platform should support multiple years of publishing and referencing between blogs. Our projects and our collaborators put effort to regularly publish articles.
  2. I shall be able to write technical articles and publish them on the same site. The articles are naturally written in Asciidoc. We want a professional looking documentation to encourage new users to try our open source components.
  3. I shall publish the technical documentation of the open source components I develop on the same site.

Hugo was selected as static site generator because its support for Asciidoc is tremendously improving.

First I used the hugo-theme-techdoc to customize Hugo. It worked great for the technical documentation and technical articles but fall short of my wishes for the blog part. When I discover the theme Docsy. It supports the technical documentation, technical articles, and blogging.

The best part is that Hugo now supports Asciidoctor natively. No more strange manipulation to load gem modules you need. And diagrams through asciidoc-diagram and plantUml are generated in the expected folders. The documentation is still on the light side but you find the needed information in the various pull requests.

The bonus is that Asciidoctor newer versions have native support for rouge syntax highlighter. It is no more necessary to load pygment highlighter and configure CSS files. Another huge gain is that plantuml and other diagrams are generated at the right location.

Install the Tools

My development platform is a MacBookPro and MacOS. I use Homebrew as a package manager.

The instructions to install hugo and asciidoctor are:

brew install hugo                         1

brew install asciidoctor                  2
gem install asciidoctor-diagram
gem install asciidoctor-rouge
gem install asciidoctor-html5s
gem install asciimath

brew install plantuml                     3
brew install graphviz

sudo npm install -D --save autoprefixer   4
sudo npm install -D --save postcss-cli
sudo npm install -D --save postcss
  1. Install hugo application
  2. Install asciidoctor and additional asciidoctor packages
  3. Install plantuml and graphviz for diagrams, in particular UML sketches
  4. Install PostCSS and additional packages used by docsy to generate deployment site. See also docsy documentation.

Asciidoctor Configuration

Below the configuration of asciidoctor in the config.toml file.

[markup.asciidocext]
  extensions = ["asciidoctor-html5s", "asciidoctor-diagram"]
  workingFolderCurrent = true
  [markup.asciidocext.attributes]
    imagesdir = "../pics"
    source-highlighter = "rouge"            1
    rouge-style = "github"                  2
    rouge-css = "style"                     3
    icons = "font"                          4
    ref-less = "https://less.works/[LeSS]"  5
  1. Select rouge as source highlighter. You should not add it to the extensions because since Asciidoctor version 2.0 rouge is included.
  2. Define the style used to highlight source code.
  3. Configure rouge to generate all formatting data into the generated html file, avoiding any css file configuration.
  4. Icons provide better looking icons for various Asciidoc} constructs.
  5. Define document attributes which are accessible to all processed documents (DRY concept for the whole site).

Docsy Configuration

Add First Level Folders

Each time you add your own first level folder - meaning at the same level as docs, blog, about, or community you need to extend the layout to support it. For example, I store technical articles in the folder ideas and use the standard template. So I need to add (if not, no items are visible in the side bar).

cp -R ./layouts/docs ./layouts/ideas
Change layouts

We had to change the partial footer.html to display a better looking copyright clause. The original version has hard coded text not really compatible with the commons creative license we are using. The layout is updated by overwriting the involved partial file.

cp $prjDir/src/site/website/docsy/layouts/partials/footer.html $siteDir/themes/docsy/layouts/partials

Enable Local Search Engine

One cool feature of Docsy is local search support through lunrjs engine.

algolia_docsearch = false

offlineSearch = true
offlineSearchSummaryLength = 200
offlineSearchMaxResults = 25

Learning

The static website is published under Open Source Components.

The source of the whole website is available under Website Source Code.

Printing of a single article is supported through your browser. You can configure printing a whole section with or without a table of contents through configuration options. For advanced cases you can define the layout of the printed document.

You can use relative links in your Asciidoc documents. Beware where the file are located by Hugo engine and the naming conventions shall follow Hugo rules.

Funny is that the blogger software and the docsy theme are from the same company Google.

This blog article is naturally written in Asciidoc syntax.

The power of Tags and Comments

The power of Tags and Comments

2020 09 01 head

A software architect models customer domains and maps them to excellent software constructs. The solution shall support future customer groups and expand to store additional information.

Tags, also called labels provide an informal ontology to classify customer data. Tags are extensible and are machine-readable.

Comments empower users to store relevant unstructured information. Comments are only for human beings.

What are good approaches to provide searchable tags and labels and avoid wild growth of useless records? Can you as a designer govern the ontology defined through labels?

Tags

Tags add semantic information to business entities. Tags are often defined as a crowd ontology without enforced schemas. Invest in domain modeling to define and maintain tags and you will move tags to a full-fledged ontology other your domain models. This ontology supports meaningful search, segmentation and reporting over historical and geographical raw data.

Tags with associated values are also used to decorate unrelated entities with orthogonal information. For example you can associate geographical coordinates - GIS - to pictures, meetings, or addresses.

A tag has three parts:

  • an optional namespace to classify the tag in the overall ontology,
  • a mandatory name to uniquely identify the tag in the context of its optional namespace,
  • an optional value providing a specific information to the tag

For example, we could define a tag such as geo:longitude=50.167958 to provide the meta information of longitude to an entity. This tag has a namespace geo, a name longitude and a double value 50.167958.

Additional information are available under link:../../docs/bus/businessmodels/ Below the source code in modern Java is

public record Tag(String namespace, @NotNull String name, String value) {}

You should strongly consider the introduction of tag types to constrain the possible values of a specific tag. For example, you can define that the namespace geo contains only the tags longitude and latitude. Both these tags have mandatory values of type double. Once your design supports tag types you can add generic validation rules in the user interface and in the business logic for all existing tags.

The ontology enforcement allows consistent reporting and data drilling over departments in your company and over time.

Comments

Comments provide human readable information and hints for entities.

An author writes a comment at a specific time. Usually comments shall be immutable. To respect human fallibility a correction mode can be supported. If changed the new comment replaces the older one and is marked as edited. The edition feature is part of the workflow and not part of the domain model.

An author can be another computer system. It can provide information about its actions through comments. If the deletion of comments is disabled, the comment history is an audit trail documenting what happened overtime with an entity instance.

Warning:

Beware that the authors define an external identifier space. If your application requires a user account, you can connect the author with the user account. The consequence is that you should only disable these accounts, and never delete them.

Policy should be defined to handle the fact that collaborators quit the company overtime and are no more active users; but they are still referenced as authors.

The code in Java is

public class Comment implements HasOid, HasTags {
    private long oid;
    private LocalDateTime created;
    private String author;
    private String text;
    private Set<Tag> tags;

    // getters and setters
}

Design

Our architecture follows the principles of domain driven design. Comments are always associated with a specific bounded domain of the overall application. For example, we often model the persistence view of a domain through a database schema. Each bounded domain has an own independent scheme and could have a persistence store for comments.

Tags are classification information associated with an instance and should be stored within the entity. The set of tags is often transformed to a textual representation and stored in a column. The search features of the underlying database can be applied for selecting instances of interest.

Tag types are a medium to define an ontology and defines valid tag instances. A set of tag types shall always be associated with a specific domain of the application.

We provide a Java library BUS implementing these constructs. More information is available under tangly open source components.

Related concepts are discussed in our blog series

Entities, Identifiers, External identifiers and Names

Entities, Identifiers, External identifiers and Names

2020 08 01 head

A software architect models customer domains and maps them to powerful software abstractions. Soon you correlate internal efficient identifiers, meaningful external identifiers and domain entities. You enjoy our long conversations with the enterprise data architect and find out the system owning a specific external identifier.

How do you relate with external systems and communicate with other companies?

How do you define internal identifiers which grow with the application success?

How can you bridge the customer world with your software solution?

What are the good practices to create a maintainable, legible and efficient model of your domain model?

Entity Concept

We need an approach to model customer domain entities and map them to a legible and maintainable software construct.

An entity is a user domain model abstraction and is mapped to a software type.

  • Any entity has a unique mandatory internal object identifier oid, we recommend the use of numeric values to improve performance.
  • Public and external identifier id, we recommend the use of text to adequately support various external identification scheme. An example is the European enterprise identifier EUID identifying a company or a certified natural person in Europe
  • Human readable name.

A more sophisticated variant provides additional features.

  • A set of tags to classify the instance through a crowd based ontology. Tags are often called labels.
  • A list of comments to add human readable information to the instance. Comments have a timeline and can be sorted by creation date. The comments can be extended to add audit information such as functional change information or activities related to the instance.

These entity features should be defined as a set of mixin interfaces.

Internal Object Identifiers oid

The object identifier oid uniquely identifies an instance in a given context. This identifier shall be a numerical value to increase performance of persistent solutions. It is used as an internal identifier in the application. It should never be visible outside the system or published through an API.

Ideally the identifier is universally valid and uniquely identifies an instance in all contexts. For example the concept of UUID tries to provide such an identifier. The drawback is that UUID is not a numerical value and cumbersome for a developer to memorize.

History has taught developers not to spare on the size of identifiers. Please use a long value meaning 64 bits.

Avoid using the internal identifier to communicate with external systems. If you respect this rule you are free to migrate your objects to another identification scheme in the future. New schemes are often helpful when the application grows, or you have later to import and take over a lot of legacy data.

External Object Identifiers id

The external unique identifier id shall uniquely identify an entity instance. It is used as an external identifier to communicate with other systems. It should always be visible outside the system and used in any API.

Try to have exactly one external identifier per object to communicate with external systems. This restriction is a corollary of the rule stating an instance is owned by exactly one system. This system is the one defining and managing the external unique identifiers.

You should clearly define the owning system for the external identification scheme. This information shall be documented and accessible to all involved parties. This process is part of the enterprise architecture activities of your ecosystem.

External identifier ownership is often a murky situation when working with legacy systems. You often have multiple source of external identifiers, sometimes overlapping and sometimes no covering all instances. You have to formulate a long-term strategy to clean-up your landscape and handle the problems until these clean-up activities are completed. By handling we mean administrative and import rules matching the various external identifiers to the same object.

One possible solution is to use tags. Store externally defined identifiers as tags. Document these tags as specific for the external system. This approach scales to multiple external identifiers, and multiple systems managing the same external identifier. So you have a scalable approach and do not pollute your domain model with spurious information defined in external systems. Upon completion of the refactoring these tags document historical information and could safely be removed.

Multiple external object identifiers imply the existence of mapping functions to identify the object referenced. Because the ownership of external identifiers is outside your system, you are dependent to these systems and have to hope they are good citizens. The key rule of good systems is that they never change external identifiers. As soon as you modify identifiers they are per definition no more real identifiers.

Names name

The entity name is a human readable name to distinguish between entity instances. Ideally it should be almost unique. The fallback is to use the external unique identifier id known to all external systems. The drawback is that we have no guarantees it is legible for users. Names are essential for well-designed user interfaces. Never require from your users to memorize external identifiers, please provide names.

For example first and last names of a person is the name for a natural entity. Social security number is a possible external identifier. An internal identifier is used as primary key in the persistence storage.

Advices

  1. Internal object identifiers are identifiers. An identifier is immutable and should be numerical for performance reasons.
  2. External object identifiers have exactly one application in charge to create them.
  3. Names are human readable and improve the legibility and usability of user interface and reports.

External identifiers can be tricky. In Switzerland, we had an old social security number which is still referenced in a lot of legal systems. For example, it is still part of your tax salary yearly form.

We have a new social security number which is used in social insurance workflows. The same number is also used in medical insurance workflows.

We have also a federal identity card number, a federal passport number, a federal driver license number, additional medical card insurance numbers, a state tax personal identification number, and more external identifiers. All these external identifiers shall referenced the same natural person.

More interesting is that a tourist living in the European zone has none of these numbers.

Please implement the internal identifier feature as an interface marker. The external identifier and name features can be grouped in one interface.

Additional information are available under link:../../docs/bus/businessmodels/. Below the source code in modern Java is:

public interface HasOid {
    long oid();
}

public interface HasId extends HasOid {
    String id();
    void id(String id);
    String name();
    void name(String name);
}

We provide a Java library BUS implementing these constructs. More information is available under tangly open source components.

Extensions

See our blog how to extend the entity concept with the powerful and flexible concepts of tags and comment approaches.

Another blog shows a constrained form of tags using the concept of reference codes also called reference tables or lookup tables.

Related concepts are discussed in our blog series

A Journey Implementing Digital Invoices

A Journey Implementing Digital Invoices

2020 07 01 head

Our company has the ambition to digitalize all our workflows since our inception. We proudly state we never bought a printer or a fax machine. Analog machines should become obsolete in the digital age. Companies should be more environment aware and save some trees. See our blog A Journey to Be a Digital Company.

Switzerland finally introduce a digital payment slip. It is based on a QR code and still assume you need to print it. The European Union is more innovative and mandates that invoices to government parties must be digital. Our company exchanges contracts and invoices with partners and customers using the standardized and well-known PDF format. It is time to explore extensions to support the new legal requirements in Switzerland and in Europe.

We show open source based solutions and the trade-offs we had to accept. Beware our solutions are based on the Java stack.

Swiss QR Code

The Swiss QR-bill standard is based on a 2D barcode and associated human readable text mirroring the data encoded in the barcode. The barcode approach assumes the invoice will be printed by the buyer. The interested parties have to scan the barcode to digitize the invoice information. This approach clearly breaks digitalization efforts.

Our approach is to add the barcode to the PDF invoice document. The workflow can open the digitally received document and parse programmatically the image of the barcode. The generation of the invoice and associated barcode is fully digital.

This approach works well only if the issuer fills the so-called SWICO field of the QR code with needed information. For example the due date of the invoice and the enumeration of different VAT tax percentages and corresponding VAT taxes are only stored in this field. In other words the standard does not foreseen that the due date of the invoice should be a mandatory field. Stated bluntly the committee in charge of the standard realized they missed and stuck all missing elements in the so-called SWICO field. It is clear that it is impossible to define a digital payment workflow without information such a due date, and the VAT percentages and amounts.

The library Swiss QR Bill is a mature and easy to use solution to read and to write legal Swiss QR codes. The library is under MIT license and is available for the Java and .NET stacks. The code is under active development and the main developer is responsive to suggestions and trouble reports.

ZUGFeRD

The ZUGFeRD is a German initiative to promote digital invoices in Germany and in Europe. The ZUGFeRD approach is to store the invoice data as a XML part in the PDF document of the invoice. The standard is also called Factur-X in France. It is compatible to European B2G invoices standard, namely EN16931. It can also be used for B2B invoices.

XML is not really a human-readable format. A JSON based approach would have been more friendly. The advantage is that the standard contains all relevant information for an invoice and streamline the digital processing of invoices. You can easily define a workflow extracting and parsing the XML payload from all received invoices, and feeding the information to your ERP system for further processing.

The library Mustang ZUGFerD is a mature and usable solution to read and write digital ZUGFeRD invoices. The library is under Apache 2.0 license. The current version 2.1 of the standard is fully supported. ZUGFeRD additionally can transform a PDF/A-1 into a PDF/A-3 format.

Other similar libraries are also available but the ones we found have quite restrictive licenses.

PDF Aspects

The PDF format is universal known. Special variants such as PDF/A1 and PDF/A3 are defined for archiving purposes.

The Apache foundation hosts the library PDFBox. It provides all low-level functions to create, read and edit PDF documents. The library is naturally under Apache 2.0 license.

But no open source products are available for high-level functions. In particular support for the creation of archive formats are non-existent. The famous iText library provides all these features but sadly is no more open source.

Ghostscript provides command line support to transform a PDF document in a PDF/A version.

The online service pdftron provides free PDF translation to archive format. The Scribus application support archive format export. The drawback is these approaches are they require manual steps [1].

The Mustang library supports the transformation of a PDF/A1 to a PDF/A3 document.

Approach

We decided to extend our open source library for creating digital invoices. The extensions support the Swiss QR Code and the ZUGFeRD 2.1 standards. All the information are stored in one PDF document. This document is sent electronically to the customer.

Our current approach is

*Generate* AsciiDoc Invoice from our invoice model (Java source code)
*Generate* PDF version of Invoice using AsciiDocPDF
    *Add* Swiss QR code as last page in the PDF invoice document
    *Add* ZUGFeRD XML payload in the PDF invoice document

Currently, we do not generate PDF/A1 documents as required in the ZUGFeRD standard. PDFBox would provide the tool box to generate in this format but it would be quite an endeavor. Ideally an extension to PDFBox with similar functionality to iText would become available in the future. Otherwise, we have to write the missing code; in particular the embedding of fonts, pictures and files into the PDF document.

We selected the AsciiDoc format to provide a simple approach to custom templates for invoices. Therefore we avoided proprietary solutions based on Microsoft Office templates.

Feel free to use our library and examples. They are released under Apache 2.0 license.

Learnings

The source code for the invoice component is available as a Java package. More information is available under tangly open source components.

The generator code for the Swiss QR bill and ZUGFeRD data is available as Java package. The developer was very supportive to improve the component. For example, it is now possible to write unit tests using in-memory file systems.

The Facture-X component also support unit tests using in-memory file systems.

Warning:

Not all Java developers have realized that the Java class File should be deprecated. java.io.File instances always access the default file system.

java.nio.Path instances contain the file system it should use and support in-memory file systems. Use java.nio.Files to create input streams, output streams, buffered readers, or buffered writers using path instances.

The investment to create the generators and test them against regular invoices of tangly llc was around four man-days.

You can find more information how to digitize your company in a blog describing our journey to a digital company. We are quite proud no having any Microsoft Office our Outlook tools. Our whole toolchain is based on open source solutions and performs flawlessly since the foundation of our company.


  1. OnlyOffice also support PDF/A creation. They provide a free local installation package for Linux, macOS, and Windows

Legacy Systems Refactoring

Legacy Systems Refactoring

2020 06 01 head

With refactoring you can take a bad design, even chaos software, and rework it into well-designed code. Most often it is cheaper refactoring a legacy application instead of rewriting it from scratch.

Each refactoring step is simple, even simplistic. Yet the cumulative effect of these small changes can radically improve the design.

Martin Fowler

Introduction

For a developer new to a legacy software solution, it is often hard to understand the existing application. He must determine the extent of source code and architectural decay, and identify smells and metric violations.

Legacy applications are often critical to the business and have been in use for years, sometimes decades. Since the business is evolving, there is constant pressure to support additional requirements. However, changing these applications is difficult and you end up spending an increasing amount of resources maintaining the software.

There are many reasons why maintaining legacy software is a difficult problem. Often, most, if not all, of the original developers are gone, and no one understands how the application is implemented. The technologies used in the application are no longer current, having been replaced by newer and more exciting technologies. Also, software complexity increases as it evolves over time as you add new requirements.

The key to managing the lifecycle of software is to continuously work as a professional and skilled developer. A professional engineer opportunistically refactor each time he modifies source code.

Understand Your Product

  • Understand how it is used. These scenarios define the acceptance tests you need to start creating a set of automated validation criteria. And you will better understand how your users are working with your product.
  • Understand how it is deployed. You need a reproducible and in the long run automated process to deploy a new version of your application. We are talking continuous delivery and even continuous deployment.
  • Understand how it is build. You need a reproducible and automated build process. This process must be integrated into your continuous integration and delivery pipeline.
  • Understand how it is structured. You need a tentative architecture description to start refactoring and to untangle this big ball of mud into a more modular solution.

Refactor Your Product

Refactoring is always a successful activity. You cannot fail. But you need discipline, continuous involvement, and measurement. If you are new to refactoring it is worth to have a coach to smooth the learning curve.

  1. Extract one big service, refactor, test, and deploy. Iterate. If you are not successful discard your failure and checkout the current working version from git. You should have learnt enough so that the next try will be successful.
  2. Refactor code. Use static analysis tools to detect the flaws in your source code. You should only improve live code, meaning code you must correct or extend. See Agile Code is Clean Code. Use a modern IDE to automate the smaller refactoring steps, and avoid spurious errors.
  3. Resolve design issues. Your senior design specialists already know the flaws. Now it is time to correct them.
  4. Increase code coverage. Code coverage is the security net when you are refactoring code.
  5. Slowly add fitness functions to continuously validate your non- functional requirements.

Beware of good practices how to write good software products and refactor successfully applications.

  • Move to Domain Driven Design as an architecture approach. It works either for micro architecture or for modular monolith approaches. Master long live domain driven design and event storming Master your technical stack and use current tools and libraries Modularize one big service extraction one after the other Avoid using dead architectural techniques. TOGAF, UML, PMI, CMMI are obsolete - various activities they recommend are good, do them in your sprints. Avoid lengthy, slow and expensive review approach. ATAM is dead. The quality tree technique is wonderful, use fitness functions to implement your quality tree. If you are running legacy technology, this not only becomes a threat to your business but also to your hiring and employer branding efforts. As fewer and fewer programmers and operation managers will have the knowledge of those systems, you’ll face a dwindling talent pool.

Ameliorate Your Process

Applications do not degrade to legacy or geriatric systems overnight. The organization and development group failed to work professionally over years before the product was ultimately doomed.

You must establish a culture of professional software development. Professional software developers write code which is maintainable and legible. Only unprofessional organizations create legacy applications.

Embrace software craftsmanship. All your developers should have formal software development training and frequent training in new approaches and techniques. They read consistently books. Is it not that you expect from your physician or the pilot of the plane you are taking?

Agile Architecture Series

The agile architecture track contains the following blogs

We also published our agile architecture course (3 ECTS) used for teaching computer science students at bachelor level at Swiss technical universities.

Gradle 6.4 Finally Supports Java Modules

Gradle 6.4 Finally Supports Java Modules

2020 05 01 head

Java 9 release introduced the concept of modules within the Java language. Java 9 was released in September 2017.

Until now creating Java modules was quite cumbersome and Gradle did only offer limited support through external plugins.

With the release of Gradle 6.4 in May 2020 software developers can finally easily generate well-behaved Java modules. We had to wait almost three years for native support!

Support for modules is native in the Java plugin. You only need to add one configuration line. Start using Java modules and have language support to enforce modularity in your product.

Java Modules in Gradle 6.4

Gradle supports the Java module system with everything you need to compile, test and execute tests for Java modules. You can also build Javadoc.

While there is some overlap with Gradle’s dependency management features, Java modules offer module boundaries that are enforced by the Java runtime. Modularity is enforced at the language level instead of coding conventions or using external tools such as ArchUnit.

The only thing you need to configure is module path inference for the Java plugins.

 java {
   modularity.inferModulePath.set(true)
 }

Then you just need to make sure you write a proper Java module by providing An example of a module information file is

 module net.tangly.fsm {
   exports net.tangly.fsm;
   exports net.tangly.fsm.dsl;
   exports net.tangly.fsm.utilities;
   requires org.slf4j;
   requires static org.jetbrains.annotations;
 }

Java 14 in Gradle Java 14 was also released in Spring 2020. It is worth to experiment with the new preview features. The following configuration is needed in Gradle to compile, test, execute and generate JavaDoc when using Java 14 preview features.

 tasks.withType(JavaCompile) {
     options.encoding = 'UTF-8'
     options.fork = true
     options.incremental = true
     options.warnings = true
     options.deprecation = true
     options.compilerArgs += ["-parameters", "--enable-preview", "-Xlint:unchecked"]
   }
   tasks.withType(Test) {
     useJUnitPlatform()
     options {
       setExcludeTags(['integration', 'localTest'] as Set)
     }
     jvmArgs += "--enable-preview"
     maxParallelForks = 8
     failFast = false
   }
   tasks.withType(Javadoc) {
     enabled = true
     javadoc.options {
       encoding("UTF-8")
       addBooleanOption('-enable-preview', true)
       addStringOption('-release', '14')
     }
   }
   tasks.withType(JavaExec) {
     jvmArgs += '--enable-preview'
   }

Beware that JaCoCo shall only release a version supporting code coverage for JDK 14 and in particular record not before June 2020.

Agile Component Design

Agile Component Design

2020 04 01 head

You are responsible to create or modify a component in your current application. This is a design activity.

How can you create a good, legible, maintainable component architecture? How can you validate your functional and non-functional requirements?

Yes you are right. You are responsible for architecture decisions at component or subsystem level. Below a set of tools to improve the quality of your design.

Design Approaches

Patterns and Idioms

The pattern movement was started last millennium. Very talented and experienced developers have documented how to solve common problems elegantly and effectively. Depending on the programming language you are using different idioms are preferred. The way to solve the same problem is different in Java, Scala, C++, Python or C#.

You should know all regular structural, behavioral and creational patterns. Explore your programming language and discover how idioms and patterns have evolved between major releases. For example lambda expressions and streams introduced in Java 8 - released Spring 2014 - completely transform the solution for regular business logic. Records introduced in Java 14 - released Spring 2020 - have a huge impact how your architecture deals with data transfer objects - DTO - and immutability.

Read Open Source Code

Stop inventing the wheel again and again. Avoid Not Invented Here syndrome. Your current problem was already solved multiple times. Explore open source solutions, read posts, study books. Select the most adequate solution and fill free to improve it.

Instead of searching for a solution for days, post your question on an adequate forum or on Stack Overflow. Again fill free to improve the suggested solutions.

Become more efficient and use the wisdom of all these developers accessible through the Internet. Always verify the quality and adequacy of their proposed solution.

Java Standard API

Know your programming language and the huge associated standard libraries part of your technology stack. Wisdom is coded in this code. Standard patterns are implemented in almost all packages. Idioms are encoded everywhere.

See how Java deals with human and machine time with the java.time package. Decades of trials and errors were needed to finally create a balanced and simple to use time abstractions. All these decisions and learnings are encoded in this code.

Clean Architecture

Clean Code

You want to create a clean architecture you are proud of. You must first write clean code. Clean architecture build up on clean code. Promote clean code in your agile team. Agile code is clean code.

Do not fall to the fallacy to draw beautiful and useless UML diagram and write thick software architecture documents. The real architecture is hidden in the source code of your product. You still should document all major design decisions.

Know Your Language

If you are developing in Java, you should use the current features of the programming language. For example with with Java 14

  • Try with resources and closeable resources,
  • Immutable collections,
  • Streams, optionals, filters, and collectors,
  • Predicates and functions to define lambda expressions,
  • Records and immutability for objects,
  • Pattern matching syntactic sugar as for instanceof operator,
  • Switch expressions Text strings.

Aggressive Refactoring

The entropy of source code increases over time. Only continuous and aggressive refactoring mitigates the degenerescence of your application. Each time you correct an error or add a new functionality refactor your code. Remove smells, compiler warnings and migrate older code to use newer and better features of your programming language.

Acceptance Test Driven Development

Your users want a working application. Write acceptance tests insuring all relevant functions are tested through your continuous integration pipeline. Therefore you guarantee your users the application behaves as specified.

Test Driven Development

Testability and changeability of your application are architectural aspects. You must have a way to verify these non-functional requirements. Test driven development is a proven approach to fulfill these requirements and validate them continuously.

Continuous Integration

Continuous integration and delivery are the mechanisms to continuously validate and verify all functional and non-functional requirements are correctly implemented. You guarantee your users and customers that any software delivery they get is compliant and correct.

Each time you find a discrepancy, add an additional test validating the requirement behind this fault. The same error will never happen again.

Good Practices

Publish your components on a central repository such as Maven Central. Your users have easy and standardized access to your components and their latest version.

Build tools such as Gradle Build Tool and Maven or IDE such as IntelliJ IDEA allows potential users to fetch the component without having to install out-of-the-box mechanisms.

Javadoc is the standard and hugely helpful approach to document classes and component public interfaces in Java. Similar tools exist for other programming languages.

Architecture design records provide hints why specific design decisions were chosen. Your users can better understand the path you follow and the selected tradeoffs of your design. They do not have to agree, but they can understand the arguments why you choose so.

Static code generator is an actual good practice to provide the current documentation and tutorials for your components. We write all our documentation in the Asciidoc format - including plantUML and highlighted source code - and generate our website using Hugo tool suite.

Start small and improve your approach every day.

Agile Architecture Series

The agile architecture track contains the following blogs

We also published our agile architecture course (3 ECTS) used for teaching computer science students at bachelor level at Swiss technical universities.

Java Records Are Here - Available in JDK 14

Java Records Are Here - Available in JDK 14

2020 03 01 head

We need justification not to use immutable objects in ours solutions, rather than an affirmative reason to use them. Records provide a first-class means for modeling data-only aggregates. They are available in Java since release of JDK 14.

Records are often seen as syntactic sugar to easily create immutable objects without having to write a lot of boilerplate code. Records as a clean definition of immutable objects can have a tremendous impact on your component design.

You as a designer have to reflect why your abstractions are not immutable and what is the rationale behind. From experience, it improves greatly the quality of your design.

Records guarantee:

  • An immutable class concept which is the strongest argument for the use of record,
  • Cannot change their state after construction,
  • Don’t change the state of the whole system in any way,
  • Don’t do side effects, like inputs and outputs for example.

Records provide

  • A set of boilerplate methods generated by the Java ecosystem. These methods are always semantically correct and efficient,
  • A public constructor initializing all instance variables Getters for all instance variables. The name of the getters is PropertyType property(). We get finally rid of the cumbersome and outdated convention PropertyType getProperty() propagated by Java Beans.
  • A semantic correct implementation of hashCode() and equals(). Therefore your objects can be used in standard API collections without difficult to find bugs.
  • A human-readable implementation toString().

The huge advantage of the approach against IDE code generation is the guaranty that these methods are semantically always correct.

Advantages

Key benefits of immutable objects are: * Thread safety * Atomicity of failure * Absence of hidden side-effects * Protection against null reference errors * Ease of caching * Prevention of identity mutation - see contract for API standard collections - * Avoidance of temporal coupling between methods * Support for referential transparency *Protection from instantiating logically-invalid objects Protection from inadvertent corruption of existing objects

Design Impact

I updated my open source projects to use the record abstraction. What a surprise! I found regularly undocumented trade-offs why a class was not implemented as an immutable type. Quite often I was just lazy and took a shortcut.

Records motivate you as a clean coder and designer to decide why a class should be mutable or not. This pressure had a tremendous impact on the quality of my subsystem design. The Java language guarantees that all record classes must be immutable.

Immutable value objects are a cornerstone in the Domain Driven Design DDD approach.

When you are using Data Transfer Objects DTO they should probably always be records. In the long run you should eliminate your DTO, they are just waste and break all the rules of abstraction and information hiding.

Tricks and Tips

You can define a constructor extension to validate the passed arguments. This validation block is automatically inserted at the beginning of the generated constructor.

 public record Range(int lo, int hi) {
    public Range {
        if (lo > hi) {
            // referring here to the implicit constructor parameters
            throw new IllegalArgumentException(String.format("(%d,%d)", lo, hi));
        }
    }
 }

As soon as the record has more than a few instance variables calling the constructor with various set of default values can become cumbersome.

Two approaches easily mitigate this problem.

  • For a simple situation create static factory methods taking care of default parameter values. For examples look at the standard API List class and the set of static of methods.
  • For more complex situation implement the builder patten to provide full flexibility to the consumers of your class.

Start using the new concept of record in your source code and implicitly improve the quality of your design. At the same time write less boilerplate code and enjoy coding.

How Agile Collaborators Learn

How Agile Collaborators Learn

2020 02 01 head

Most products are developed by a team, and these achievements are the fruit of teamwork.

Software product development is now agile, and most often uses the Scrum approach.

Just putting a bunch of people together in the same open office will not produce a high performance agile team. Quite a few companies find that out the hard way.

How can you increase the odds to create an environment where agile teams bloom and perform?

Attitude versus Aptitudes

Your business is nothing more than the collective energy and efforts of the people working with and for you.

If you want to make your business better, invest in your people. They’ll get the job done.

Learn and pass on good development practices through pair programming, mob programming and coding dojos. Promote Communities of Practice CoP.

It is not enough management commit themselves to quality and productivity, they must know what it is they must do.

Such a responsibility cannot be delegated.

W. Edwards Deming

You have hired good people. Train them to improve and perform better.

Continuous Learning and Improvement

CFO to CEO: “What happens if we invest in developing our people, and they leave us?”

CEO to CFO: “What happens if we don’t, and they stay?”

Peter Baeklund

Secure collective knowledge of the code through code reviews, pair collaboration or collective work. Avoid any developer working solely on his own on a specific piece of code. Crafting successful software applications is teamwork. Encourage your collaborators to write blogs and ask questions in forums such as Stack Overflow.

Establish together development standards and keep them up to date. Nurture these good practices through Communities of Practice CoP.

Rely on tools such as continuous integration, static metrics and architecture fitness functions to shorten the feedback loop. See our post about pragmatic software craftsmanship.

Establish a culture to use current versions of programming languages, tools and libraries.

On a previous project, we had planned a coding dojo ritual once every two weeks, during which we shared our practices with some perspective over the product. It was an occasion to experiment with new technologies, assess their match for the project needs, share new coding techniques, and update our standards together.

Formal Training

Professional software developers should have formal training in software development. You should expect from a person working forty hours a week for the next forty years to pursue a bachelor or a master in computer science.

You should expect regular certifications in the used technology stack or development approaches. A Scrum master two days training does not make somebody an expert. A specialist refusing to invest in a two days training is probably not a member you want in a high performing team.

A professional engineer should read multiple technical books each year.

Leadership Responsibilities

Create psychological safety in the workplace.

2020 02 01 psychological safety

The major responsibility of senior management is hire the best collaborators, retain and develop top collaborators, and respectfully off-board departing collaborators.

  • Hiring and on-boarding collaborators,
  • Nurturing and development of collaborators,
  • Off-boarding of collaborators,
  • Move collaborators to the learning zone and let them thrive.

Nurture a culture of learning through training on the job, learning from external sources - such as reading high quality blogs - and formal education - such as bachelor or master courses for example agile architecture -.

Please reflect how you encourage continuous learning in your organization.

Training is always an investment and never a cost point. Look how you budget and account training activities. Are they really investment positions?

Agile Architecture Series

The agile architecture track contains the following blogs

We also published our agile architecture course (3 ECTS) used for teaching computer science students at bachelor level at Swiss technical universities.

Calibrate Your Agile Approach: Bill of Rights

Calibrate Your Agile Approach: Bill of Rights

[width=420

When eXtreme Programming first came out, one expectation it worked hard to set is who gets to make what calls on software projects.

For example, business has the right to know how long something is going to take and when it can expect things to be done.

Conversely, developers have the right to know what the priorities are, and to be able to do good work.

To that end, eXtreme Programming created the Customer and Development Team Bill of Rights.

Customer Bill of Rights

You have the right

  • to an overall plan, to know what can be accomplished when and at what cost.
  • to get the most possible value out of every programming week.
  • to see progress in a running system, proven to work by passing repeatable tests that you specify.
  • to change your mind, to substitute functionality, and to change priorities without paying exorbitant costs.
  • to be informed of schedule changes, in time to choose how to reduce the scope to restore the original date.
  • to cancel at any time and be left with a useful working system reflecting investment to date.

Developer Bill of Rights

You have the right

  • to know what is needed, with clear declarations of priority.
  • to produce quality work at all times.
  • to ask for and receive help from peers, managers, and customers.
  • to make and update your own estimates.
  • to accept your responsibilities instead of having them assigned to you.

Calibration

You shall use the above bill of rights to evolve the cooperation between the development teams and the customers and improve the product value. See if:

  1. Your customer has an overall plan with scheduled functionalities and costs. Your development team can schedule changes and update the estimates,
  2. Your customer has at any time a running and tested application. Your development team always produce quality work,
  3. Your customer can cancel at any time the development initiative. Your development team receives help from peers, managers and customers,
  4. Your customer and the development team accept responsibilities.

Dysfunctions are easy to detect

  1. Customers are not involved in the development initiative on a daily basis,
  2. Development team does not pursue craftsmanship and quality,
  3. Customers cannot cancel initiative at any time, Development team cannot update estimates,
  4. Management force responsibilities upon collaborators.

Observe your organization and start to improve. The above rules are here to detect Agile Bullshit and truly improve your organization toward building products which delight the customers.

2019

A Journey to be a Digital Company - tangly llc

A Journey to be a Digital Company - tangly llc

2019 12 01 head

I long dreamed to work in a digital environment. I shall no longer print out a form, sign it by hand and use snake mail to deliver it to the receiving party. All company documents are stored electronically. I can search any artifacts by content and get a list of its in seconds.

Our software service company is an ideal candidate to validate if this assumption is realistic.

Profit is like the air we breathe. We need air to live, but we don’t live to breathe.

Frederic Laloux

What is Digital Working?

You shall use digital tools and workflows to perform all your work related tasks. We want to increase interactions with our coworkers and remove any tedious activities.

  • Fax is dead. You do not need faxes in Europe - no more waiting at the fax machine,
  • You send your contracts electronically. You can sign them electronically - no more printing or paper archiving,
  • You send invoices electronically - no more printing,
  • You perform your accounting electronically and avoid any paper archive,
  • You coordinate with your team members with chat tools; not over the phone or with emails. N o more answering long emails with huge cc list and less business travel,
  • Any document can collaboratively be edited online by multiple collaborators working distributively. No more calls to find the latest version of a document or to remind somebody to unlock the document. Or worse finding out why your changes just disappeared,
  • Any collaborator has access to all our assets. You monitor all software continuous integration and delivery pipelines from anywhere if authorized. The information is available with a regular notebook or on any mobile device.

And you have a positive impact on the environment. Less traveling, less paper and less shipping. Embrace the digital age and be environment aware.

All Activities and Processes are Digitized

Internal Tools

At tangly we use internally

  • Google Business Apps and LibreOffice for office work, email, calendar, video conference and document sharing. All documents are stored in our product or company online drives,
  • Google Business Apps administration console to manage our technical infrastructure such as workstations, mobile devices, DNS record,
  • Bitbucket and continuous integration, delivery and deployment pipelines for software development,
  • Trello for activities management and our electronic Scrum or Kanban board. We use analog boards if the team members are collocated - we are trying to avoid JIRA, for us it is just overkill -,
  • Slack is our preferred chat platform,
  • Banana is our accounting solution.

We never had to bought or rent a printer, a landline or a fax machine since founding the company.

External Tools

We use an impressive set of tools to collaborate with our customers and partners

  • Google email, calendar and meetup for communication,
  • Bitbucket and continuous integration, delivery and deployment pipelines for software development,
  • Maven Central to distribute public binary software artifacts,
  • Slack is our preferred chat platform,
  • Google groups for technical discussion,
  • Blogger to publish articles to our community,
  • LinkedIn for marketing,
  • GPG and S/MIME to securely exchange confidential documentation or protect sensitive emails.

An impressive set of processes with governmental organizations can now be performed electronically in Switzerland

  • VAT authority support declaration and payment electronically,
  • Company register in Switzerland give access to all registered companies,
  • Salary declarations and forms are generated electronically through government online tools,
  • Swiss post office support package tracking and delivery rescheduling online,
  • Swiss social insurances - pension fund, children allowance, military service, unemployment insurance - declaration and mutation are available online,
  • Taxes declaration can be performed with a downloadable government application,
  • Public transportation company tickets can be bought online and delivered electronically on collaborator’s mobile device.

What is Working?

  • We perform marketing and sales through online platforms and avoid any physical ads platform,
  • We communicate through our company online platform. We never use physical documents and minimize email usage,
  • We deliver all contractual documents electronically,
  • We deliver all invoices electronically,
  • We have our library and technical documentation as static web sites, wikis, and electronic documents. We do not anymore possess paper books,
  • Our archive is online. We do not possess a physical archive of documents or contractual documents and never bought or rented physical space for this purpose,
  • We never did own or rent any printer, land phone line or fax machine. The quality and speed of our Internet connect improve yearly. It is already above our keenest expectations.

What is NOT Working?

Surprisingly most of the processes with private companies providing mandatory services cannot be done electronically due to obsolete information systems and cumbersome processes.

  • Accident insurance (BU and NBU) still request declaration of incidents and communication over paper mail,
  • Pension fund insurance company provides a web portal but still send some forms over paper mail,
  • Tax declaration must still be printed out and sent to the tax office of the state government.

None of the government solutions provide modern API. We still must manually input our data in their portals. The old interfaces they provide are only partially available through expensive commercial solutions. By old interfaces we mean proprietary formats and proprietary file transfer solutions.

Findings

The last four years were a game changer. A lot of manual activities are now fully digitized and often automatized. We never had to buy a printer. The last year we seldom had to scan a paper document.

The partners still requesting paper interactions are some government agencies and some insurances. We observe a decrease of such physical interactions. Our partners are steadily - if sometimes slowly - digitizing their workflows.

Slowly digital versions for invoices are emerging. This approach should allow automated payment workflows. Interestingly private customers already have billing and e-payment solutions. Similar solutions for corporate customers are either not available or incredibly limited, cumbersome and expensive.

The next big step will be the emergence of API to integrate the steps in a workflow specific to the involved companies. For example the technical part of the onboarding process for a new employee could be fully automated.

Agile @ Scale

Agile @ Scale

2019 11 01 head

Your organization has decided to introduce agile at scale in all development departments or better in the whole company.

Your Chief of Agility COA, Chief of Digitalization COD or Chief of Change COC was in an expensive training. After a few drinks he has selected the appropriate framework for your company.

Welcome to the club of companies introducing agile approaches and having no clues why they do it and how they could be successful.

Here some hardly learnt truths worth knowing when starting such an endeavor.

Imposing Agile methods introduces a conflict with the values and principles that underlie Agile methods.

Martin Fowler

The scaling approaches I encounter at customer sites in Switzerland and Europe are

  • Spotify Model was popularized by Henrik Kniberg. His YouTube videos are worth watching. The banking group ING adopted the approach and made it fashionable.
  • Scrum@Scale defined by Jeff Sutherland and Scrum Alliance. Henrik Kniberg has written an interesting article stating Spotify Model is a kind of Scrum@Scale.
  • Nexus defined by Ken Schwaber and Scrum.org.
  • LeSS defined by Craig Larman and Bas Vodde. It is also supported by Scrum Alliance.
  • SAFe 5.0 defined by the RUP aficionados and often trashed by Kniberg, Sutherland, Schwaber, Larmann and Vodde. At the same it is the most popular approach in quite a few countries. And I would never state it is the most successful.

Additional approaches exist but I did not encounter them lately.

Delay Scaling

The key learning is

Delay your Scaling. First Lean the ropes. Try to descale your processes.

Learn the ropes with small agile teams applying Scrum, Kanban, eXtreme Programming or our own approach. Learn as much as you can through experiments before considering scaling. The invested capital in small experiments with small teams is tremendously cheaper than experimenting with company-wide approaches.

Be gentle and kind with your professional teams. Scale once your agile teams are seasoned.

It takes time to learn and master Scrum and Kanban approaches.

Recognize that organizational agility is constrained by technical agility. Focus the first years on technical excellence, software craftsmanship and clean development.

All the above mentioned methods emphasize the importance of technical excellence for durable success.

Simplify your processes, roles and structures. Reflect if you really need scaling.

Now you are ready to experiment with scaling.

Understand the Principles of Scaling

  • Create a learning organization. It is the building block for agile approaches
    • Train your collaborators,
    • Experiment with new approaches and measure success,
  • Organize around up to eight teams, one backlog for one product
    • Avoid team backlogs,
    • Have one product owner in charge of the whole product,
  • Pull, do not push
    • Collaborators and teams select workload,
    • All information are available to all collaborators,
  • DevOps is king
    • Automate aggressively,
    • Put all artifacts under version control,
  • Focus on outcomes and impact, not on outputs
    • Customer features are product only when used by users,
    • Velocity and burn-down charts are output.

Avoid Common Errors

Study the history and evolution of the various scaling approaches. In particular the changes in SAFe are an archaeological treasury what they wrongly stated and how they corrected some of the identified flaws.

  • Continuous delivery is the new approach. Eliminate hardening iterations, integration phases, milestones when software should be integrated. Your teams shall deliver a workable software on demand, multiple times a week or a day.
  • Organize around products. Customers buy products. Eliminate product based organizations. *b Have one backlog for the whole product. Otherwise multiple backlogs must be synchronized and local optimization is de facto applied. This approach is against all lean principles.
  • Customer and users talks directly with the development team. Product owners, product managers enterprise architects should never be filtering the information flow between customers and engineers.
  • Eliminate step by step all coordination roles. Emphasize communication between developers in the team and between teams. Scrum of Scrums where only Scrum masters attend, Release Train Engineer, System teams, corporate Solution Engineers are just a waste of resources.
  • Realize technical excellence is the only approach to deliver quality products to the customer. Raise the importance of technical excellence, and never forget that when writing software, the technology side is really vital.

Final Words

Establish agile teams. Thin your process. Chose your scaling approach. Try it and measure impact. Iterate and improve continuously. Therefore, it could be necessary to change your initial scaling approach and adapt it to your company needs.

I wish you successful scaling of agile approaches. And I have to warn you the path to success is long and risky. Personally I had some successes with Large Scale Scrum - LeSS.

Time to Migrate to a Modern JDK, Laggards!

Time to Migrate to a Modern JDK, Laggards!

2019 10 01 head

Are you still using Java 8 to run your mission critical systems?

Are you truly using a six years old technology stack?

Are you already applying for a Java laggard club membership?

If you are still using Java 8 you are using technologies released in Spring 2014. You are missing all the improvements in the garbage collector, runtime engine and various API.

Java 11 LTS was released in September 2018. The current LTS release is one year old. You had one year to migrate to this long term solution version.

It is time to modernize the fundament of your application and invest a small amount of effort in it. The payback is fantastic.

Advantages

  • You have economical gains through
    • Major performance improvements make your application faster. You have either higher customer satisfaction or need less processing resources,
    • Major garbage collector updates make your application more predictable. The maximum amount of memory is often lower,
    • Better behavior in container environment translates to less resource usage,
  • You have more legible and maintainable source code
    • Since Java 8 useful new language features have been introduced, along with new tooling,
    • Various standard API were extended with convenience methods,
    • One important change was that internal APIs - largely those classes in packages that started with sun.misc.* - were hidden from use,
    • APIs that are not core to the JDK have also been removed in Java 11 or later. These changes may impact your application but there is a clear path to avoid these problems.
  • You have to select either the long term support path or upgrade every six months the Java runtime
    • When upgrading the choice you face is whether to use the latest version of Java, currently 13 and be prepared to upgrade every six months. Or upgrade to the latest LTS 11 to give yourself up to three years to think about your next upgrade,
    • Don’t be tempted to ignore compiler warnings. Deprecation is being taken much more seriously in this modern Java world, and both Java 10 and Java 11 removed APIs.
    • In less than two years the next LTS will be released with Java 17. It is scheduled in September 2021.
  • Become more agile
    • Once over this first upgrade, it is worth at least testing the application on the latest version of Java every 6 months, for example in CI.
    • In all cases openJDK is now the new default. Various companies - Oracle, Redhat, Amazon, Azul, IBM - provide commercial support for openJDK for different timelines. You are more flexible with your migration timeline.
2019 10 01 jdk releases

How to Migrate?

You do not need to implement Java modules to migrate to Java 9 or beyond. It is worth the effort to slowly support the module approach. Your architecture will become more modular and the interfaces are more clearly defined.

There are basically four incremental phases to fully migrate to Java 11 or later:

  • Run an existing Java application with the JDK,
  • Compile the application with the JDK,
  • Use the new features of the current JDK,
  • Modularize the application to use module system.

You could recompile and run your solution with the new Java version without code changes.

Changes are necessary if you use deprecated packages or access Sun internal packages. All deprecated packages have a compatible alternate implementation. Often you just need to update the import statements. Sun packages have a documented migration path to alternate approaches providing similar functionality. By the way Sun company already stated years ago you should not use these internal packages. So time to clean-up your code.

Call for Action

Start now your migration to a modern JDK and leave Java 8 behind you.

Anyway you can only tinker with the date, you will have to migrate at some point.

A Commons View on Scrum

A Commons View on Scrum

2019 09 01 head

Agile approaches encourage common ownership of artifacts during product development. Historically commons is the term used for shared resources. Can we apply the commons learnings to agile and Scrum approaches?

Interestingly economists were kind to state common ownership is doomed to fail through the tragedy of commons theory published end of the sixties.

Thirty years later Elinor Ostrom showed that commons can indeed work well if you follow a small set of rules. She found century old examples scattered around the world. She was awarded the Nobel Prize in economics for her findings in 2009 and dissipated the previous fake news.

The picture shows a Suone in Wallis, Switzerland. The constructions bring water to arid regions and are build and maintained by communities. It is example of commons in place for hundred of years and is one of the concrete implementation studied by Elinor. Mountain pastures are also managed as commons in Switzerland. You will find a drawing of Suone on the hundred Swiss Francs note.

A resource arrangement that works in practice can work in theory.

Elinor Ostrom identified eight design principles of stable local common pool resource management through the world. These principles have a wider range of application than common resource pool groups. They are relevant to nearly any situation where people - such as Scrum teams - must cooperate and coordinate to achieve shared goals.

Below the eight principles are presented with a mapping to Scrum aspects.

A clear definition of the contents of the common pool resource and effective exclusion of external un-entitled parties

The Scrum teams define the artifacts and processes exclusively owned by the team. Teams take easily ownership of items such as source code, Scrum board, pull process. Other items such as refactoring process, incident tickets produce more infighting. You can measure the maturity of your team accordingly to the clarity which resources they own exclusively. The first step shall often be collective ownership of source code and common coding style.

The appropriation and provision of common resources that are adapted to local conditions

Agile teams own commons resources such as * Collective ownership of source code, * Definition of Done DoD, * Sprint Backlog, * Coding Guidelines, * Team rules and work techniques.

The work techniques often limit common resources in inexperienced teams. They ask the product owner if they could refactor a class or invest a few minutes in clean code techniques. They do not own the internal quality of the product. More mature teams are able to make the transition with the support of their Scrum master and the organization. They truly own the source code and its quality.

Collective-choice arrangements that allow most resource appropriators to participate in the decision-making process

Scrum or LeSS have quite a few platforms to support decision making process * Daily Scrum, * Retrospective, * Review, * Backlog refinement, * Planning.

The decision-making process works only if the organization grants psychological security to their teams. In my experience it takes years until an organization discards command and control reflexes and delegates responsibility and accountability to the Scrum team members. Often daily Scrum are reporting meetings - either to SM or PO or to the development team - and seldom a platform to collaborate - discussing options and experimenting -.

Consensus or consent approaches are widely more successful than majority decisions is one of the findings of Elinor.

Effective monitoring by monitors who are part of or accountable to the appropriators

You have to monitor your commons to know if they are healthy. You can

  • Pair program or review commits with pull requests,
  • Automate static metrics and test coverage,
  • Implement continuous integration, delivery and deployment,
  • Try zero bug policy.

I regularly state monitoring is the first commitment of team members to be publicly accountable. Often it is painful to realize how difficult transparency is.

A scale of graduated sanctions for resource appropriators who violate community rules

Rules are only respected if sanctions are implemented upon violations of the agreement. Scrum teams can rule that

  • You must repair the broken build,
  • You must immediately correct your coding violations,
  • You loose your source code management system check-in rights,
  • You are excluded from team.

Most teams need counseling before they can tackle with the concept of sanctions. As a Scrum master you must gently empower them to sanction. If this rule is not implemented you will always land in the tragedy of commons and utterly fail in your agile journey. Worse your product will probably also fail.

Mechanisms of conflict resolution that are cheap and of easy access

Conflict resolution shall be fast, cheap, and timely. Scrum provides excellent approaches

  • Automated checks on the source code and executable application,
  • Daily Scrum,
  • Review and retrospective.

The automatic checks are worth the effort as an effective, neutral, and cost effective to detect violations and automatically block the offender. The Scrum events are platforms to discuss and resolve the discovered violations. The Scrum master must facilitate the discussion until the team members have developed their own conflict resolution instruments.

Self-determination of the community recognized by higher-level authorities

Self-determination works only if recognized by the overall authorities and organization. Here we leave the team level and need department recognition - for a LeSS approach - or company level recognition - for example to have ownership to remove a team member -.

  • Self-organizing of Scrum team,
  • Ownership of internal quality,
  • Ownership of estimations.

Scrum master shall coach and counsel the organization and the team. It takes time until management understand the dependencies between delegation, accountability, ownership, and autonomy. And you shall remember Larman’s Laws

Culture follows structure.

Craig Larman

As a change agent you will together with leaders change the structure of your organization. Please be gentle and patient.

In the case of larger common-pool resources, organization isin the form of multiple layers of nested enterprises. Small local CPRs at the base level.

Scaling agile practices at the organization level require multiple levels.

  • Transparency through Scrum board,
  • Definition of Dome as contract between team and organization,
  • Visibility of source code, continuous integration, delivery and deployment of artifacts,
  • Scale to product level using LeSS.

If you are ready to scale up to the company you could consider Beyond Budgeting Round Table. BBRT and Sociocracy approaches and tailor them to your specific needs.

I rediscovered the commons rules through a presentation of Craig Larmann at the LeSS conference 2019 in Munich. He inspired us to look at agile approaches through the commons' lens.

Agile Architecture within Scrum

Agile Architecture within Scrum

2019 08 01 head

You are applying the agile, lean approach to your software product development. Your teams use Scrum or Kanban to deliver running software every two weeks.

Agile architecture is a key element to improve the quality of your article and reduce cycle-time of your product outcomes.

Your journey toward greater business agility starts by identifying what outcomes are most important to your company’s success. A key component is to find out how to create great architecture within the Scrum framework.

What Is Agile Architecture?

An agile architecture shall have four characteristics.

2019 08 01 architecture
  1. Allow change quickly because we cannot foreseen the future
  2. Always verifiable because we want changes without impeding integrity
  3. Support rapid development because we want effective and efficient implementation of new features
  4. Always working because we want continuous deployment to have customers' feedback -

These characteristics are essential for successful agile development of great software products.

What Is The Scrum Approach?

Scrum is silent how architecture shall be performed during sprints. We can use the Scrum values, the Scrum guide and the agile manifesto principles to infer it.

The architect shall be part of the Scrum development team. This is the most embedded way an architect would fit into a Scrum team. It may not even be an individual who has the title of architect. The big idea behind Scrum teams is that team members have different skills and their roles may change to deliver what is required for a particular sprint.

So it could be that senior developers within the team got architecture skills, or are in the best position, or have the best knowledge, to make those architectural decisions. Those team members with architectural skills are involved in the sprint on a day-to-day basis. They attend the daily Scrum meetings, take items off the backlog, and work on them.

Interestingly LeSS is more opinionated and describe successful approaches for agile architecture and design.

How Do You Perform Agile Design?

Scrum is silent about technical practices. The Scrum fathers and community strongly emphasize to adopt eXtreme Programming practices. For bigger systems use the good practices described in LeSS.

An actual very good practice is to use event storming and domain driven design DDD to model the application domain and partition the system.

Use fitness functions to monitor and validate continuously all non-functional requirements.

The well-established practices from the object-oriented and DevOps communities are Develop very early a working skeleton, Clean architecture implies clean code and requires clean coders in your team (see our post clean code), Aggressive refactoring and understand technical debt, Continuous integration, continuous delivery, and continuous deployment. For a detailed discussion consult the post Pragmatic Craftsmanship.

What is the Architect Role?

The architect shall

  • Be part of the team, work daily with team members and attend all team events,
  • Maintain the pace of development,
  • Often Communicate and early as a coach and mentor - meaning pair programming and design sessions -,
  • Be fluent in domain driven development, hexagonal architecture, refactoring, and clean code,
  • Use architecture design record ADR and light documentation to describe architecture decisions - meaning no lengthy Software Architecture Document SAD or overwhelming UML models -,
  • Be a master programmer and writes code in the application technological stack. He writes actual source code as current good practices are,
  • Have rich knowledge of architecture and design.

We have mentioned refactoring, which is how we evolve an ever-improving design and architecture while building the product incrementally. To do that, we need to know the difference between good architecture and design and not-so-good. We need a rich pool of architecture and design options in our bag of tricks, ranging from very simple options to the more complex and rich options that we may need as the system grows. One never knows too much about the quality of architecture, code, and design. One does, of course, need to apply that knowledge smoothly, incrementally, little by little.

Agile Architecture Series

The agile architecture track contains the following blogs

We also published our agile architecture course (3 ECTS) used for teaching computer science students at bachelor level at Swiss technical universities.

The World is a Better Place and We Must Tackle Climate Change

The World is a Better Place and We Must Tackle Climate Change

2019 07 01 head

The world has really improved in a lot of areas.

Stop whining we are doomed. And yes we must act strong and fast to tackle the climate change disaster.

But do not forget how much the world bettered the last centuries.

We are not doomed, we must solve the next challenges instead of just complaining and doing nothing.

  • Ask yourself if you need a car.
  • Travel with train instead of flying.
  • Schedule a video conference instead of a business trip.
  • Reduce your plastic and paper usage.
  • Each step reduces your carbon footprint.

Millions of persons taking similar steps will be a huge improvement. Just do it!

The late Swedish academic Hans Rosling has identified a worrying trend. Not only do many people across advanced economies have no idea that the world is becoming a much better place, but they actually even think the opposite. This is no wonder, when the news focuses on reporting catastrophes, terrorist attacks, wars and famines.

Who wants to hear about the fact that every day some 200'000 people around the world are lifted above the $2-a-day poverty line? Or that more than 300'000 people a day get access to electricity and clean water for the first time every day? These stories of people in low-income countries simply does not make for exciting news coverage.

What is better?

Extreme poverty is plumping down

2019 07 01 world poverty

https://ourworldindata.org/extreme-poverty

Literacy and education is exploding

2019 07 01 world literacy

https://ourworldindata.org/literacy

Health is improving and child mortality is crashing

2019 07 01 world life expectancy

https://ourworldindata.org/life-expectancy

Please stop the doomsday trashing and start to act to improve your social environment and diminish your solution footprint. I am tired hearing we cannot improve our world. Stop lying and simply look at the facts. Feel free to argument based on facts.

I recognize that we have difficult problems to solve and success is not guaranteed. I am just tired of negative thinking and persons claiming we cannot improve our world.

Start learning and explore the data on Our World In Data. They have interactive charts.

Sources

https://www.forbes.com/sites/stevedenning/2017/11/30/why-the-world-is-getting-better-why-hardly-anyone-knows-it/#2ab7ae478268

Seven Reasons the World is Improving

23 charts and maps that show the world is getting much, much better

Research and interactive data visualizations to understand the world’s largest problems

What do the people of the world die from?

And the beautiful book from Hans Rosling

https://www.amazon.com/Factfulness-Reasons-World-Things-Better-ebook/dp/B0756J1LLV/ref=sr_1_1?keywords=hans+rosling&qid=1563089194&s=digital-text&sr=1-1

Agile Code is Clean Code!

Agile Code is Clean Code!

2019 06 01 head

You are applying the agile, lean approach to your software product development. Your teams use Scrum or Kanban to deliver running software every two weeks.

To create an agile architecture you must write agile worthy code. And agile code is always clean code.

Start early, do it continuously and soon your code will be clean, legible and free of defects.

Architecture versus Code

Be honest. Having a huge class with thousands of lines of code plainly demonstrate that your organization worked unprofessionally over months in the past. Such monsters are not born over one night coding session. It takes time and incompetence to create such maintenance nightmares.

And as long as your product contains such code monsters it is worthless to pretend having a nice and thought through architecture.

Team discipline and early warning systems are the key to avoid such blunders and kill the monster in the egg.

Clean Code and Refactoring

The clean code test driven development TDD approach to write code was defined last millennium. Professional developers adopted this approach to deliver high quality, legible and maintainable code artifacts.

2019 06 01 red green refactor
  1. Red - Write a test and run it, validating the test harness,
  2. Green - Write the minimum amount of code to implement the functionality validated in the previously developed test. Update the code and run the tests until the test suite shows successful execution,
  3. Refactor the code to reach clean code level. Continuously run the tests to guaranty compliant changes.

If you are a Java developer you will * Use JUnit 5 and Mockito 2 to implement the first step, * Use a powerful IDE such IntelliJ IDEA to write and enhance the functional code, * Use tools such as SonarLint, Jacoco, Spotbugs, ArchUnit to refactor your functional code and test cases. Clean code applies to productive and test code.

The described approach is a good practice since the beginning of this millennium. All above mentioned tools are open source and free to use. Your organization has no excuse to write crap code.

Just write well engineered, legible and maintainable source code. It is a matter of discipline. It is also a prerequisite to agile architecture.

Acceptance Test Driven Development

How do you know what are the requirements of a specific feature? Either write huge amount of outdated documentation or find a better way. The better way is acceptance test driven development ATDD.

For each feature you need acceptance criteria; this approach is also called specification by example. For each acceptance criteria write an acceptance test which is executed each time the application is updated in the continuous delivery pipeline. Therefore the acceptance tests are the living documentation of the behavior of your product.

To implement this approach you need

  • Acceptance criteria which imply specifications by example - you need to train your business analysts and product owners,
  • Cucumber for Java (or equivalent tools) - to write acceptance tests, Mock all external systems and the database - to have fast build and test time and avoid external dependencies,
  • A continuous build and delivery pipeline to automatically run your acceptance validation tests upon each application change,
  • Avoid user interface tests - because they are slow and brittle,
  • Use in memory database only if you have to - they are a fast solution to write acceptance tests but are intrinsically slower than mocks.

Code Sociology

The most simple engineering practices are the most powerful. Practice consolidates changes in behavior of your development organization.

  • Pair Programming - is a training on the job and crowd wisdom approach to steadily improve knowledge and quality -,
  • TDD and ATDD - are safety net to guaranty quality attributes in your source code,
  • DevOps starting with git, docker, continuous integration, delivery and deployment - to speed up development and eliminate tedious manual errors,
  • Lack of accountability equates to lack of trust - your teams are accountable for the quality of your product and you should trust them.

Once good practices are established you can move to more sophisticated tools - such as git history code analysis - to uncover weaknesses in your development department.

Teams ready to learn from the like of Google could evaluate Trunk Based Development and consider long-lived branches as waste.

Code Review

A code review can be done different ways. Many teams use GitHub, Bitbucket or GitLab. So a very common approach for our code reviews are pull requests. However, there are situations where pull requests are not necessary. For example, if a feature is implemented in pair programming or mob programming then the code is already reviewed while it is written.

We decided that it does not have to be reviewed again but of course, everybody may look at the commits later on in GitHub and add comments. As a rule of thumb we agreed that every line of code has to be approved by at least one other developer before it is pushed into our master branch.

Pull Requests in Trunk Based Development

In trunk based development it is different. Since we want to merge our commits into the master branch as quickly as possible, we cannot wait until the complete feature is finished. Unlike in the original trunk based development approach we still use feature branches but we have much less divergence from the master branch than in Git Flow. We create a pull request as soon as the first commit is pushed into the feature branch. Of course that requires that no commit breaks anything or causes tests to fail. Remember that unfinished features can always be disabled with feature toggles.

Now, with part of the new feature committed and the pull request created, another developer from the team can review it. In most cases that does not happen immediately because the developers don’t want to interrupt their work every time a team member pushes a commit. Instead, the code reviews are done when another developer is open for it. Meanwhile, the pull request might grow by a few commits.

The code is not always reviewed immediately after the commit but in most cases it reaches the master branch much quicker than in Git Flow.

Agile Architecture Series

The agile architecture track contains the following blogs

We also published our agile architecture course (3 ECTS) used for teaching computer science students at bachelor level at Swiss technical universities.

Agile Architecture Principles

Agile Architecture Principles

2019 05 01 head

You are applying the agile, lean approach to your software product development. Your teams use Scrum or Kanban to deliver running software every two weeks. The product market introduction is a success and you will successfully sell the solution for the next ten years.

How shall you define the software architecture of your product and insure a long living and high quality solution? The experts tell you the design is emergent, what does it means?

LeSS Architecture Observations

The following observations are true for any software product. It is irrelevant if developed using agile approaches or traditional older ones.

  1. The sum of all source code is the true design blueprint or software architecture,
  2. The real software architecture evolves (better or worse) every day of the product, as people do programming,
  3. The real living architecture needs to be grown every day through acts of programming by master programmers,
  4. A software architect who is not in touch with the evolving source code of the product is out of touch with reality,
  5. Every programmer is some kind of architect — whether wanted or not. Every act of programming is some kind of architectural act — good or bad, small or large, intended or not.

Agile architecture comes from the behavior of agile architecting

  • hands-on master-programmer architects, a culture of excellence in code,
  • ban emphasis on pair-programming coaching for high-quality code/design,
  • agile modeling design workshops,
  • test-driven development and refactoring,
  • and other hands-on-the-code behaviors.

Quality of Your Architecture

You shall measure the quality of your design and produced software artifacts.

  • Static analysis tools - validate your source code -,
  • Test driven development - validate your design -,
  • Acceptance test driven development - validate your functional requirements -,
  • Fitness functions - validate your non-functional requirements -,
  • Pair programming - improve the produced artifacts through wisdom of the crowd -,
  • Pair review and pull requests - validate your developers' work -.

The above validation activities are backed through tools. But these tools can only detect non-quality and never provide a proof you achieved quality. Qualified and experienced engineers is your best guaranty to achieve and sustain high quality in architecture and software products.

Implement the above measures to gather data and publish the actual state of your source code. You find further information in Code Scene as Crime Scene, SonarLint for the Impatient, and Pragmatic Craftsmanship articles.

Read the LeSS architecture page for a comprehensive discussion of agile architecting.

Good versus bad architecture

A good architecture fulfills the specifications and is easy to change.

It shall emerge during the development and intentionally implement the known requirements.

Your architects are talented developers and are full members of your Scrum teams.

Your development teams * are expert in the used programming language and stack, * understand object-oriented, functional and rule based programming, * known all major patterns and idioms of the used development stack, * practice TDD, ATDD, clean code, refactoring, embrace CI/CD and DevOps, * read source code from open source projects to learn better ways, * know SMART, INVEST, SOLID, KISS, YAGNI, * hold weekly design workshops with huge whiteboards, * use domain driven design and event storming, * avoid BDUF.

You shall keep it simple, make it valuable, and build it piece by piece.

The above hints and practices shall empower your teams to practice successfully agile architecture and timely deliver delightful software solutions. Your organization shall train your collaborators, see also Shu Ha Ri model. Smart money goes in training your collaborators.

Agile Architecture Series

The agile architecture track contains the following blogs

We also published our agile architecture course (3 ECTS) used for teaching computer science students at bachelor level at Swiss technical universities.

Why I still love to use a MacBookPro and MacOS in 2019"

Why I still love to use a MacBookPro and macOS in 2019

2019 04 02 head

As a young developer I loved Linux, compiled new kernels during evening sessions and struggled days to have the correct drivers for the graphic card and communication components of my notebook.

I grew older and decided to enjoy my weekends, family and outdoor activities.

And surely I shall have no virus, trojan and other evils on my workstation. So I went to macOS and Apple notebooks without regrets.

Gains

The major gains using macOS are * Still no virus scanner is necessary, the speed-up during complex programming and development activities is tremendous, * No trouble when updating OS, updates are automatic and often no new start is required, Unix command line and tools are available in the console, Homebrew provide all known and less-known utilities and programs available under Linux. * My previous MacBook is now six years old and still used as a development backup platform. The performance is still very good, * My preferred IDE Intellij IDEA works flawlessly on macOS, * My new MacBookPro is perfect for Java development, database integration and running multiple Docker images locally. The 32 GB RAM goes a long way to run multiple docker images and databases.

No virus scanner is still important. I was lately at a customer site and the whole deployment just crashed because IT department decided to update the virus scanner and associated options. Suddenly the generated artifacts were considered harmful. This decision just broke the CI/CD pipeline for hours.

Homebrew easily downloads and installs newer versions of favorite applications and utilities. It solved the dread of macOS slow or missing application updates. Pre-installed Unix utilities are seldom kept up to date by Apple. Just use Homebrew and forget about Apple slow updates.

Daily Development

The tools I really enjoy and use on a daily basis for software development - mainly Java - are

  • IntelliJ IDEA IDE is still a dream. It could sometimes be slightly snappier but still the best Java IDE under the sky,
    • UML diagrams are created with plantUML and the provided plugin,
    • Documents are written in Asciidoc with the existing plugin,
    • Git is kept up to date with homebrew,
    • Gradle Build Tool is kept up to date either with homebrew or gradlew,
  • Atlassian Cloud applications - BitBucket, Trello and CI pipeline - and SourceTree for some git operations,
  • Homebrew as package manager for utilities,
  • Docker as container manager - for complex product you can run locally kubernetes,
  • Chrome is my default browser. I almost never use _Safari or Firefox,

I retired VirtualBox because I do not need any virtual machines. They were replaced with Docker images, I stopped using MacPorts. Homebrew is more than enough for my needs.

Daily Work

The tools I used to perform administrative work are * LibreOffice - going strong and better every year -, * Google Business for teamwork in the cloud using collaborative tools, * Apple Mail Client with GPG plugin for PGP and S/MIME secure email, * A local Swiss article Banana for accounting and VAT reports for the federal government. I bought it as soon as the company stopped requesting higher prices for macOS than for the other platforms.

I still have no need to use any Microsoft applications. My findings are inline with the ones in my older blog in March 2016. I hope Apple will still produce high-quality hardware and software in the future.

The only big macOS improvement I am waiting for from Apple is that Apple delegate the installation and update of Unix applications and utilities to Homebrew. Apple is sadly always behind the release cycle of regular Unix programs. Homebrew does a way better job to support software developers.

Code Scene as Crime Scene

Code Scene as Crime Scene

2019 04 01 head

You have a successful product and happy users. Slowly the cost adding new features is creeping up and product margins are shrinking. Which crimes did put us in this dreaded situation?

How can you analyze the history of your product’s source code?

How can you explore the social dimension of your product development? How can you find good approaches to increase time to market and improve development costs?

This article presents a set of tools you can use to better understand the alternatives and select the best approach for your product.

Our approach is based on the de facto standard for source code version management git. If your development department is using another tool you should consider moving to another company.

First Steps

We want to investigate various contexts in our source code history and identify suspects. A context can be a single file, a class, a package or a module.

Gather overall information concerning your repository - use the optionals --before and --after to restrict the time range you are interested in -

git log --numstat --pretty=format:'[%h] %an %ad %s' --date=short --before='YYY-MM-DD' --after='YYYY-MM-DD'

Find out which contexts are most often modified

git log --pretty=format: --name-only | sort | uniq -c | sort -rg | head -100

or find out which contexts with a specific extension are most often modified

git log --pretty=format: --name-only | sort | uniq -c | sort -rg | grep -I extension | head -100

Find out how many times a specific context was modified

git log -- contextPath --pretty=format:'%an' | grep 'Author' | sed -e 's/Author: \(.*\) <\(.*\)/\1/' | sort -rg | uniq -c | sort -rg

Find out how many errors were fixed in a specific context - e.g. identified through close #TicketId -

git log -- contextPath | grep 'close #' | wc -l

Deeper Insights

2019 04 01 code that matters Analysis of various products found out that 5% to 10% of the source code is under active development. It is where your money is spent to make your customers happy. Just put all your refactoring efforts to improve these 10% of your product currently impacting your customer satisfaction.

Technical debt reduction should always be prioritized to only these hotspots. Simply use the above described approach to identify the most often changed files during a specific time interval.

Identify team and social metrics

To understand team dynamics you should group individual developers to their team. Simply replace the name of the team member with the team name at the command line using grep or sed. Now you can analyze the set of files the team mainly modifies and maintains. If none can be found you have a diffuse code ownership and will have quality issues.

Using the domain driven design DDD terminology, bounded domains are developed and maintain through one team. This approach encourages architectural purity, continuous refactoring, and accountability.

Manage Off-boarding Risks

Each time you find a set of source code artifacts only modified by a single individual you have identified a major off-boarding risk. When this developer will leave the department or the company all knowledge associated with the development and the maintenance of these components will immediately be lost.

Visualize

The above scripts provide information in textual forms. Over time you will become more sophisticated with your inquiries and need better tools. Either visualize your findings with d3js or plotly.js or buy a commercial tool such as code scene.

Another approach is to write a small framework to analyze the data and to implement more complex queries in your preferred environment.

Next Steps

The above techniques are part of the toolbox of professional development departments. Establish a software craftsmanship culture in your company. It helps you to avoid the invasion of gangs and eradicate crime in your neighborhood.

Find similar ideas in our blogs Pragmatic Craftsmanship, SonarLint for the impatient, and You need an Engineering Culture.

Two books published in the pragmatic programmer’s series are a wonderful deep analysis of source code as a crime scene [1] and scanning your application source code [2].

Literature

  • [1] Your Code as a Crime Scene: Using Forensic Techniques to Arrest Defects, Bottlenecks, and Bad Design in your Programs, Adam Tornhill, 2015
  • [2] Software Design X-Rays, Adam Tornhill, 2018

You Need an Engineering Culture

You Need an Engineering Culture

2019 03 01 head

Agile approaches are not enough, You need a strong engineering culture to achieve quality and lasting success.

Often coaches and Scrum masters tinker with the processes and invest effort and blood to improve the ceremonies and the company structure.

But at the end of the day the success of your application is defined by its adequacy to customers' needs and its overall quality. You need a solid and professional engineering culture to produce high quality products and release them at high rate. If not you just release a new set of defects to your paying customers.

How do you establish and nurture a professional and effective engineering culture?

Live and Breath Your Values

Remember, values drive your culture and the daily behavior of all your collaborators. But even if you have invested time and blood to identify the right values and put all the mechanisms in place, it might still not have the desired results. Culture is something that happens daily through personal interactions. Putting the right values and mechanisms in place will help move your company in the right direction, but it’s not a guaranty for success.

In order to increase your odds for success you need to live and breath your values. You need to truly believe at all levels in the organization and not compromise on them.

  • Advocate for build-in quality,
  • Support employee first,
  • Promote technical excellence,
  • Don’t be penny wise and pound foolish.

Communicating

Make sure that people know the values. It should be part of every on-boarding. It is one of the first things that new people need to learn. But also make sure to remind people as often as possible, e.g. in employee guidelines, during appraisals and town hall meetings.

Eat Your Own Dog Food

People look up to you and they will imitate your behavior. So make sure that you and all members of the leadership team are the evangelists and provide at all times the right example.

Read our blog post Pragmatic Craftsmanship for hints what a technical excellence culture should be.

Learn from the LeSS] community.

You can find an initial list of questions in our blog post Company Culture Check.

Validating

Validate that all teams are living up to their values. Find a way to measure it. Identify and put actions in place in case you need to improve. If a value doesn’t make sense, e.g. the value was defined early on but in hindsight it doesn’t fit with your company, then get rid of it.

Last Thoughts

Building a great engineering culture is not something that you easily can do. You cannot cheat. It will not automatically happen by copying the values and organization structure from another company such as Spotify. You need to really invest in creating a culture that works for your company.

You can look at organizational structures, values and principles from other companies to get inspired. But most important sit down with your people and decide together what is important for your company. Put mechanisms in place that will support the concepts, allow them to grow. Then live and breathe it within all levels of your company.

How Healthy is Your Product? - Company Culture Check

How Healthy is Your Product? - Company Culture Check

[width=420

You are developing your application using agile and lean approaches.

How can you check your approach and distill improvements? A health check of your product and your development approach is certainly a good solution.

This post is the fourth and last of a set of articles identifying health checks with different focus. We will identify strengths, potential weaknesses and hopefully find room for improvement.

What is your company culture and how does it focus collaborators to delight your customers.

Rules

The patient solely decides if a health check shall be performed. The effort and depth of a health check is defined together with the patient. The findings are considered confidential.

Many attempts at achieving business agility are misguided and fake. But I consider the glass half full, not half empty.

Fake Agile is a signal to do better. Fake agile is a treatable disorder.

Jurgen Appelo

A short check with a feedback workshop presenting the insights requires around two work days. A deeper and more intensive check requests more effort, between one and four weeks. Upon completion, you can also hire a coach to implement selected findings and improve durably the health of your application. The findings are often clear and the measures straightforward. The real work is to implement consistently the measures.

I have the same personal challenge when trying to lose weight or improve my running form. The long term implementation is the crux.

Company Culture Checks

The biggest flag to identify a learning and improvement focused culture is perhaps this statement

I get better and lasting improvements with organizations who invest in training their collaborators.

Not only training on the job, but they send them to courses, exhibitions and buy them books.

A set of questions to detect cultural flaws for a lean and agile focused company are

  1. Do you still have individual bonuses?
  2. Do the organization provides feedback to each employee at least on a monthly basis?
  3. How complex is your travel expenses regulation?
  4. Are all C-level managers trained and certified in lean agile leadership and agile approaches?
  5. Are the human resources department collaborators trained in agile approaches and agile HR?
  6. Are all collaborators and managers working in the development organization trained in agile and lean approaches?
  7. Are all collaborators and managers working in the development organization trained in software development techniques?
  8. Is there a product charter that lays out the mission and strategic goals?
  9. Do all members of the team understand both, and are they able to see how their work contributes to both?
  10. Do you release your products at least monthly?
  11. Do software developers have local administrator rights on their workstation?
  12. Can software developers install tools and libraries on their workstation?
  13. How is the working environment of your talented developers?
    1. Do they have a team area with a dedicated desk for each team member?
    2. Do they have multiple high-resolution displays to work with - high-resolution means 4K or higher -?
    3. Do they have root access - admin rights - to their development workstation?
    4. Do they have access to technical books and articles?
    5. Do they have height adjustable desk?
    6. Do they have big whiteboard and flip charts to work on in the team area?
    7. Do they have rooms to meet and work as a team in the vicinity and available?
    8. Do they use the best development tools of trade - and not only the basic and cheap version -?
    9. Do they have access to fast and stable continuous integration and deployment pipelines?
    10. Do they develop and document in English?
  14. Can you work on refactoring and clean code without a product backlog item or task?
  15. Do your managers knows the most important sentences and practice them daily?
    1. Sorry
    2. Thank you
    3. I am sorry
    4. Can I help you?

Posts in the Health Check Series

Detecting Agile Bullshit

Detecting Agile Bullshit

[width=420

The Defense Innovation Board DIB advises the secretary of defense of the United States of America.

Members of the board are recognizable names like Eric Schmidt who was CEO of Google and Alphabet, or Instagram COO Marne Levine.

So I was quite surprised that a government committee was so straightforward to publish a document called Detecting Agile Bullshit [1].

Simple questions are used as a bullshit detector to identify organizations faking agile. Worth your time, if the government has this problem, you probably will also encounter it in the private sector.

Tips to Identify Fake Agile Organizations

Many attempts at achieving business agility are misguided and fake. But I consider the glass half full, not half empty.

Fake Agile is a signal to do better. Fake agile is a treatable disorder.

Jurgen Appelo

Agile development has become such a holy grail that some organizations or product teams wave the agile flag whether they are true believers or not. They throw around talk of sprints, stories and Scrum without meaning it. Signs of this behavior are

Ignoring users
  • Nobody on the software development team is talking with and observing the users of the software in action. We mean the actual users of the actual code,
  • Continuous feedback from users to the development team (bug reports, users assessments) is not available. Talking once at the beginning of a program to verify requirements does not count!,
Not deploying early an initial product release
  • Meeting requirements is treated as more important than getting something useful into the field as quickly as possible,
Development teams are silos instead of feature teams
  • Participants act like ‘it’s not my job’,
  • Technical Excellence
    • DevOps or DevSecOps culture is lacking if manual processes are tolerated. Such processes can and should be automated (e.g. automated testing, continuous integration, continuous delivery),
    • Distributed version control management systems - Git - are not used,
    • Continuous integration and delivery - Jenkins - are not used,
    • Modern deployment platforms - Docker, Kubernetes - are not used.

Questions to Ask Programming Teams

  • How do you test your code?
    Wrong answer: “we have a testing organization responsible for testing”
    • What tool suite are you using for unit tests, regression testing, functional tests, security scans, and deployment certification?
  • How automated are your development, testing, security, and deployment pipelines?
    • What tool suite are you using for continuous integration CI, continuous deployment CD, regression testing, program documentation?
    • Is your infrastructure defined by code?
  • Who are your users and how are you interacting with them?
    • What mechanisms are you using to get direct feedback from your users?
    • What tool suite are you using for issue reporting and tracking?
    • How do you allocate issues to programming teams?
    • How to you inform users that their issues are being addressed and/or have been resolved?
  • What is your (current and future) cycle time for releases to your users?
    • What software platforms to you support? Are you using containers? What configuration management tools do you use?

For a team working on agile, the answer to all of these questions should be a form of “yes”.

Questions for Program Management

  • How many programmers are part of the organizations that owns the budget and milestones for the program?
    Wrong answers: “we don’t know,” “zero,” “it depends on how you define a programmer”,
  • What are your management metrics for development and operations; how are they used to inform priorities, detect problems?
  • How often are they accessed and used by leadership?
  • What have you learned in your past three sprint cycles and what did you do about it?
    Wrong answers: “what’s a sprint cycle?,” “we are waiting to get approval from management”,
  • Who are the users that you deliver value to each sprint cycle? Can we talk to them?
    Wrong answers: “we don’t directly deploy our code to users”.

For a team working on agile, the answer to all of these questions should be a form of “yes.”

Questions for Customers and Users

  • How do you communicate with the developers?
  • Did they observe your relevant teams working and ask questions that indicated a deep understanding of your needs?
  • When is the last time they sat with you and talked about features you would like to see implemented?
  • How do you send in suggestions for new features or report issues or bugs in the code?
  • What type of feedback do you get to your requests/reports?
  • Are you ever asked to try prototypes of new software features and observed using them?
  • What is the time it takes for a requested feature to show up in the application?

For a team working on agile, the answer to all of these questions should be a form of “yes”.

Questions for Program Leadership

  • Are teams delivering working software to at least some subset of real users every iteration and gathering feedback?
  • Is there a product charter that lays out the mission and strategic goals? Do all members of the team understand both, and are they able to see how their work contributes to both?
  • Is feedback from users turned into concrete work items for sprint teams on timelines shorter than one month?
  • Are teams empowered to change the requirements based on user feedback?
  • Are teams empowered to change their process based on what they learn?
  • Is the full ecosystem of your product agile? Agile programming teams followed by linear, bureaucratic deployment is a failure.

For a team truly working agile, the answer to all of these questions should be a form of “yes”.

Conclusion

The above questions are taken directly from the document Detecting Agile Bullshit. Evaluate organization to find out if they or you are agile.

Read my related set of blogs How Healthy is Your Product?

Additional blogs for an in-depth check of your agile framework, values and current work processes are:

Now government procurement acknowledges that some companies are just cheating with their agile claims, improve yours before getting caught. Luckily the check will find out you are really being agile instead of pretending.

I wish good luck and success with your agile transformation.

Literature

2018

How Healthy is Your Product? - Product Strategy Check

How Healthy is Your Product? - Product Strategy Check

2018 12 01 head

You are developing your product using agile and lean approaches.

How can you check your approach and distill improvements? A health check of your product and your development approach is certainly a good solution.

This post is the third of a set of articles identifying health checks with different focus. We will identify strengths, potential weaknesses and hopefully find room for improvement.

What is your product strategy and how do you involve your customers and stakeholders.

Rules

The patient solely decides if a health check shall be performed. The effort and depth of a health check is defined together with the patient. The findings are considered confidential.

A short check with a feedback workshop presenting the insights requires around two work days.

A deeper and more intensive check requests more effort, between one and four weeks. Upon completion you can also hire a coach to implement selected findings and improve durably the health of your application. The findings are often clear and the measures straightforward. The real work is to implement consistently the measures.

I have the same personal challenge when trying to loose weight or improve my running form. The long term implementation is the crux.

Product Strategy Checks

Vision of the product
You can communicate the vision on your product in an elevator pitch and have it hanging in the team workplace.
Roadmap of the product
You have a visible roadmap describing the major releases for the next 12 to 18 months. Define the right planning horizon for your article.
Release plan of the product
You have a release plan for the next 6 to 9 months.
Story map
You have a story map grouping the stories and epics to reflect your release plan.
Customer involvement
You have customer interaction in every sprint. Not only during the review event, but also during the planning meeting and architecture discussion.
Review with customer involvement
Realized stories are discussed during the review and demonstration meeting and timely feedback is provided to allow backlog refinement before the next planning meeting. A mature organization has acceptance criteria for each story and automated verification of the criteria at any time.
Early and regular delivery of an increment of the product
You get feedback on the delivered and installed product increment. This goal is only achievable if your organization has implemented {url-articles}2018/blog-2018-11-01[Delivery Pipeline Check],
Risk List
You have a documented risk lists with description of risks, probability of occurrence and cost per event, and mitigation measures,
Solution architecture
You have an emergent solution architecture with prototype and support the team to refine the architecture proposal and to implement it in the solution. Do not forget an agile product needs an agile architecture. See also the LeSS approach for creating an agile architecture.
Non functional requirements
You have identified your non functional requirements and constraints and have fitness measures in place. The measures guarantee the continuous achievement of the constraints - often defined as -ility requirements -.

See the {url-articles}2018/blog-2018-11-01[Delivery Pipeline Check] to enable your organization to provide realtime feedback how well the above constraints are correctly implemented in the product.

Posts in the Health Check Series

SAFE 4.6: Improvements Direly Needed

SAFE 4.6: Improvements Direly Needed

2018 11 02 head

It is official. The power engine of SAFe 4.6 vehicle is Team and Technical Agility. View the official presentation and more important download the slides. It is one of their five Lean Enterprise competencies and one of their four core values. SAFe finally recognizes the tremendous importance of technical proficiency and craftsmanship.

They advocate two findings. First you need team agility, how you apply Scrum or Kanban. In other words being agile instead of doing agile. Second you need technical agility, how you develop a quality product.

I challenge you. If you are following SAFe, talk with your teams and checks how many of their official good practices are you using on a daily basis.

I have seen SAFe introductions in Swiss, German and Czech companies. Attention was set on the structure of the release train and a lot of effort was spent on the various ceremonies, meetings and artifacts. But how to produce quality software and apply technical excellence was sadly always an afterthought. And more than once a senior SAFe coach just publicly stated that the SAFe core value x is not so important for a successful introduction. Quite a denial approach of his own methodology I think!

Be honest with yourself and your teams, either you implement the techniques described in https://v46.scaledagileframework .com/team-and-technical-agility/[Technical Agility] and Build-In Quality chapters of SAFe or you are not agile and have no SAFe power engine. It is called cargo cult.

A good starting point are the five dimensions of build-in quality described in SAFe 4.6. And yes now SAFe acknowledged the tremendous importance of a continuous delivery pipeline. Experts agree you can apply DevOps only if you have continuous integration, delivery and deployment.

Once you realize where your organization stands remember that build-in quality is one of tour core values of SAFe. That I mean is walk the talk.

And please read some Internet blogs and books about LeSS, Software Craftsmanship, Extreme Programming, TDD, ATTD, clean code. Most of these concepts were defined and published in the last millennium. It is time your teams finally adopt them and produce high quality, maintainable and cost-effective products. This is the whole concept of the lean movement. And I welcome the fact that SAFe now explicitly mention Cost of Delay.

One of the strengths of SAFe is their clear statement that an agile organization transformation is only successful with lean-agile leadership. All C-level managers are trained in lean and agile, they advocates these approaches and are visible champions. Therefore all managers shall be advocate technical excellence on a daily basis.

If not the seminal work of John Kotter clearly states the transformation will fail. You will have a bunch of Scrum teams in your development department, but have none of the expected synergies at product or company level.

Personally I prefer the LeSS approach, but I respect your choice - SAFe, Nexus, or Scrum@Scale - as long as technical excellence is a key aspect of your agile strategy. Embrace technical agility and do not let your train of teams produce mediocre software every two weeks.

Employ professional software craftsmen and deliver quality products. Follow your SAFe approach, respect one of the four core values and one of the five core competencies of your method.

See also my blogs {url-articles}2018/blog-2018-04-01[Pragmatic Craftsmanship], {url-articles}2018/blog-2018-06-01[Software Quality Graal], and How healthy is your Product? article series for information about technical excellence.

The other big improvement is that the PI horizon is now limited to 8 or 12 weeks. No more semesterly or hopefully quarterly cadence.

It still does no make sense to plan for three months and at the same time advocate DevOps and design thinking.

DevOps has a time horizon of days, it is waste to have huge quarterly PI gatherings and plans just to discard them after the next DevOps finding. You should probably just update your PI roadmap.

Call for Action

My call for action is fewer meetings and paper artifacts and more technical excellence. You want to sell excellent and well-designed products, not mediocre software.

How Healthy is Your Product? - Delivery Pipeline Check

How Healthy is Your Product? - Delivery Pipeline Check

2018 11 01 head

You are developing your product using agile and lean approaches.

How can you check your approach and distill improvements? A health check of your product and your development approach is certainly a good solution.

This post is the second of a set of products identifying health checks with different focus. We will identify strengths, potential weaknesses and hopefully find room for improvement.

How you build and deliver your software product has direct implication on your liquidity and cash flow. Improvements in your delivery pipeline have direct impact on your finances. === Rules

The patient solely decides if a health check shall be performed. The effort and depth of a health check is defined together with the patient. The findings are considered confidential.

A short check with a feedback workshop presenting the insights requires around two work days.

A deeper and more intensive check requests more effort, between one and four weeks. Upon completion, you can also hire a coach to implement selected findings and improve durably the health of your application. The findings are often clear and the measures straightforward. The real work is to implement consistently the measures.

I have the same personal challenge when trying to loose weight or improve my running form. The long term implementation is the crux.

Delivery Pipeline Checks

How reproducible, fast and economic is your product deliver pipeline? At the end of the day you deliver regularly a reproducible and trackable product version to your customers.

One nice aspect about delivery pipeline checks is that a whole set of tools and methods emerged with the DevOps movement. You still need an expert to tune the diagnostic to your company specific needs but the approach is well documented.

Below a list of capabilities we have build-up over time for delivery pipeline checks. Use common sense and strategic goals to identify the objectives you want to measure. Expertise is needed to define values to measure if the source code of your product reaches the selected objectives.

  1. Continuous integration pipeline with quality checks and automatic tests,
  2. Continuous delivery pipeline packaging the same binary to all your environments,
  3. Continuous deployment if adequate for your solution, certainly it is adequate for the test and integration stages,
  4. Automatic source code quality checks and mandatory quality gates,
  5. Automatic unit tests, integration tests and end to end tests,
  6. Golden Trunk in Git,
  7. No branches or only short-lived branches policy,
  8. Zero bug policy - eliminates the change management processes and board for errors -,
  9. Traceability of specifications to delivered features and product versions,
  10. Semantic versioning and API documentation,
  11. Configuration and infrastructure as managed artifacts,
  12. Generation of documentation part of the delivery pipeline,
  13. Monitoring and alarming,
  14. DevOps pipeline analysis provides another look at your processes
    1. Code development and Review, Continuous integration and build status,
    2. Continuous testings and business risks feedback,
    3. Artifact repository and pre-deployment staging,
    4. Release automation including change management and release approvals,
    5. Infrastructure configuration and management and infrastructure as code,
    6. Applications performance monitoring and end-user experience.

You should know how often you can deliver your product without failures and track your lead time between fixes. The overall goal is faster time to market and improved delivery frequency.

The overall features and processes can be implemented through your development organization is less than a year. Stop evaluating, start implementing!

Continuous Integration Practices

Check your process to see which continuous practices you are really mastering

  • Maintain a single source repository - no long-lived branches
  • Automate the build - no manual inputs, no manual configuration, automatically triggered
  • Make your build self testing - TDD, ATDD, static metrics, semantic checks
  • Everyone commits to trunk every day
  • Every commit should build trunk on an integration machine
  • Fix broken build immediately - lean red button stopping production, everybody helps fix the broken build Keep the build fast - We are talking of minutes not hours
  • Test in an integration staging area which is a clone of the production environment - the goal is to have docker images, virtual network, and in-memory database to be fast
  • Make it easy for anyone to get the latest packaged release
  • Everyone can see the build status - transparency, information radiator
  • Automate deployment

Posts in the Health Check Series

How Should You handle Definition of Done?

How Should You handle Definition of Done?

2018 10 02 head

An agile team is sole responsible for the internal quality of the product they build and maintain.

And the developers shall be accountable to produce the expected quality and optimize long terms goals of the organization. They provide the answer to the seminal question

Did we build it right?

Definition of Done DoD is a major building block to answer this question with a sounding yes.

Definition of Done

Scrum and LeSS have a stringent discussion concerning Definition of Done. The key question is

_Considering our current context and capability, what activities can be completed each Sprint?

This subset is the initial Definition of Done.

A Definition of Done is weak when it is a small subset and strong when it is almost equals to Potentially Shippable.

In huger organizations the development teams discuss their context and select the subset of the activities that all teams think they realistically can do during the Sprint. This is their initial Definition of Done. The teams that can do more will expand this product Definition of Done within their members.

The difference between the Definition of Done and Potentially Shippable is referred to as Undone Work.

Potentially Shippable - Definition of Done = Undone Work

The Sprint is planned according to the Definition of Done and thus the Undone Work is excluded. It is planned to be left undone.

The terms Potentially Shippable and Definition of Done are often not used consistently. To clarify the terms:

Potentially Shippable
All activities that must be performed before the product can be shipped._
Definition of Done
An agreement between the teams and their Product Owner on which activities are performed inside the Sprint. A Definition of Done is perfect when it equals to Potentially Shippable. Teams strive to improve towards a ideal Definition of Done.
Undone Work
The difference between the Definition of Done and Potentially Shippable. When the Definition of Done is perfect then there is no Undone Work. If this isn’t the case then the organization has to decide, first how do we deal with the Undone Work, and second how do we improve so that there is less Undone Work in the future.
Unfinished, not finished, or not done—Work
that was planned in a Sprint but was not completed. This is often confused with Undone Work.
Unfinished
is work that the team planned for but did not finish whereas Undone Work was never even planned for. When a team has work that was not finished then they ought to feel anxious and discuss improvement actions during their retrospective.

Teams should never leave work-in-progress at the end of the Sprint and “carry over” to the next one. This causes a lack of transparency and reduces scope flexibility. If they forecast too much work, they need to remove complete items which they haven’t started yet.

Items of a Definition of Done

  • A delivery standard as defined by the team,
  • It contains all requirements to get a user story into production,
  • The fitness for use is evident (enough value was built to justify releasing?),
  • The external quality is verified (we have built the right thing?),
  • The internal quality is verified, you wrote unit tests – and all tests are green,
  • The code is checked in, if necessary the branch was closed and deleted,
  • The code review was completed,
  • All improvements from the code review were implemented,
  • All existing unit tests remain green,
  • The acceptance tests were verified by the development team,
  • All integration tests were passed,
  • and the Status of “done” was confirmed by the product owner.

The following criteria are often overlooked and also need to be checked:

  • Was the technical documentation updated?
  • Was the user documentation updated?
  • Was the user documentation localized?
  • The localization for the application is done.
  • The localization testing is done.
  • The marketing input is done.
  • The legal documents are done.
  • The deployment and migration scripts are available.

The extension of definition of done to deliver a potentially shippable product has often significant and profound impact on the structure of the organization and its processes. Experienced agile coaches use the definition of done as an instrument for organizational changes.

When the undone work is slowly removed from the delivery process it triggers structural and process changes. For example the separate quality department responsible for the final tests is dissolved and their expertise is integrated in the development teams.

See also the blog Pragmatic Craftsmanship for a discussion of build-in quality.

How Healthy is Your Product? - Source Code Check

How Healthy is Your Product? - Source Code Check

2018 10 01 head

You are developing your product using agile and lean approaches.

How can you check your approach and distill improvements? A health check of your product and your development approach is certainly a good solution.

This post is the first of a set of articles identifying health checks with different focus. We will identify strengths, potential weaknesses and hopefully find room for improvement.

The initial examination shall always be the source code. Your code base is a key asset of your company.

Rules

The patient solely decides if a health check shall be performed. The effort and depth of a health check is defined together with the patient. The findings are considered confidential.

A short check with a feedback workshop presenting the insights requires around two work days.

A deeper and more intensive check requests more effort, between one and four weeks. Upon completion you can also hire a coach to implement selected findings and improve durably the health of your application. The findings are often clear and the measures straightforward. The real work is to implement consistently the measures.

I have the same personal challenge when trying to loose weight or improve my running form. The long term implementation is the crux.

Source Code Checks

How clean, legible and maintainable is your source code? Often it is best to start your health evaluation with your product source code. At the end of the day it is compiled source code you deliver to all your customers to work with.

One nice aspect about source code checks is that a whole set of tools are available on the market. You still need an expert to provide the diagnostic but the gathering of data can be automated and quite inexpensive to gather.

Below a list of capabilities we have build-up over time for source code checks. Use common sense and strategic goals to identify the objectives you want to measure. Expertise is needed to define values to measure if the source code of your product reaches the selected objectives.

  1. Java 11/10/9/8/7/6 code and how to migrate to a current version and use the new Java features,
  2. Multi-threaded, parallel programming and reactive systems,
  3. Functional and object oriented detailed design approaches,
  4. Test driven development TDD and build-in quality - gradle or maven -,
  5. Acceptance test driven development ATDD and user acceptance,
  6. Clean code approach and detection of code smells,
  7. Static analysis tools to measure the quality attributes of source code,
  8. Compilation and test automation,
  9. Software craftsmanship approach, values and techniques,
  10. Domain driven development and micro services,
  11. Database models and open data,
  12. Software architecture and Enterprise architecture,
  13. CI and CD - continuous integration, continuous delivery, continuous deployment,
  14. Static quality assurance approaches for quick diagnostics,
  15. Development tools to improve quality and productivity,
  16. User Experience and UI design.

My blogs Pragmatic Craftsmanship and How to Reach the Software Quality Graal provides insights why source code quality is of tremendous importance for any company.

And the blog {url-articles}2018/blog-2018-05-01[SonarLint] for the Impatient would be a pragmatic first measure to measure and improve your source code health.

Any product should have a battery of measures deployed to continuously monitor the source code and detects trends over time.

Posts in the Health Check Series

Introduce Agile in A Company

Introduce Agile in A Company: Shu Ha Ri

2018 09 01 head

First teams have successfully used Scrum to develop applications. The news spread out and everybody wants to apply Scrum. It is time to introduce Enterprise wide Agile with Scrum or Kanban or XP.

Your Company wants to introduce Enterprise Agility in their product development, quality assurance and sales departments. The company as a whole shall transition to an agile organization and all teams shall adopt Scrum.

Try the Shu Ha Ri approach for such a change process.

Based on our exhaustive experience we recommend starting with a set of workshops with the goal to define a tailored approach to adapt the structure and processes of the company and introduce enterprise agile.

These workshops provide the platform to identify the company’s needs and strategic goals, and gather support from all participants. Try the open space format to involve all collaborators and identify their needs.

Below we provide an approach which must be later tailored to the specific needs of the Company during the above mentioned workshops. In particular the organizational needs for feature oriented planning and development of a portfolio of applications should be considered.

Approach

We work with the Shu Ha Ri agile approach when introducing enterprise agile in the Company. All collaborators will progress from the novice level through the proficient level before reaching the expert level. You can also use the agile fluency model if you prefer.

This model also maps very well to the “Awareness – Desire – Knowledge – Ability – Reinforcement “or ADFAR model. The Company has already reached the awareness and desire level because they want to introduce enterprise agile. Below we describe the activities for the development organization consisting of the software development, product management and quality insurance departments, and the activities at company level.

Development Organization

Shu - Novice Level - „Learn the Rules“

The duration of the novice level is at least six months. The major activities are to align the collaborators and provide the needed knowledge through training and hands-on exercises. The extensive training is aligned with experiences and industrial best practices.

All collaborators shall attend an introduction to agile and Scrum approaches. All technical collaborators shall attend the Scrum - Master or Developer - training. Each group of technical collaborators will learn specific techniques and skills necessary to work effectively in an agile product environment.

The agile coaches have two roles. First they facilitate and support the teams to implement all meetings, artifacts, and rules of the Scrum approach. Second they train the team members how to use the learnt techniques such as clean code, pair programming, pair check-in, agile architecture, requirements elicitation, test driven development, agile quality assurance, acceptance test driven development, continuous integration, deployment and delivery.

After the initial training the coaches ensure that this knowledge and ground rules are applied in daily activities.

DeveloperBusiness Analyst / POSM / PL / Line ManagerQuality Manager / TesterOthers
Agile and Scrum Introduction(1 day)
Scrum Master (2 days)
Product Owner (2 days)
  • agile and clean developer
  • TDD
  • ATTD
  • Agile Development Game
  • DVCS – Git –
  • Legacy code refactoring
  • Clean Code
  • Agile architecture and patterns such a inversion of Control
  • Agile Requirement Management
  • Lean Startup
  • Lean UX
  • Design Thinking
  • ATDD
  • Agile Leadership
  • Advanced Lean
  • Agile Quality
  • Soft skills – coaching, mentoring -
  • Agile Quality Management
  • ATDD
  • Move them to a Scrum team
?
~ 20 days~ 10 days8 days8 days1 day

We recommend

  1. An Agile (eXtreme Programming techniques) and Scrum coach responsible of two to three Scrum teams.
  2. Coach of coaches for the growth of internal agile and Scrum coaches for the company.

Ha – Proficient Level - “Bend the Rules”

The duration of the “proficient level” will be up to two years. The goal is to reinforce the learnt techniques and become proficient in using them. T he motto is

Most difficulties in Scrum don’t arise from learning the Scrum framework. Rather they arise in applying the framework on an existing organizational structure and culture.

This requires hands-on experienced guidance, and not just at the team level, but at the leadership and organizational level. See below the discussion about enterprise transformation.

We recommend

  1. An Agile (eXtreme Programming techniques) and Scrum coach responsible of four to five Scrum teams.
  2. Two coaches of coaches for the growth of internal Scrum coaches for the company.
  3. Start to blend the team coaching with the company wide changes. See below the chapter “Organization Transformation”.
  4. Workshops could be organized for selected products or teams to solve identified major impediments.

Ri – Expert Level - “Break the Rules”

The “expert level” will be reached earliest after two years. It is reasonable to state that not all collaborators will reach this level of expertise. The goal is to transfer ownership of the agile approaches to Company collaborators. The company will take over the responsibility to reinforce agile and Scrum activities and resolve newly identified impediments in products or at company level. The motto is

*Doing* agile is easy. Being agile is hard.

We recommend

  1. An Agile and Scrum coach responsible for six to eight Scrum teams. The internal coaches should start taking responsibilities
  2. A center of competence for the growth of internal Scrum coaches for the company.

Company Transformation

While teams implement Scrum, managers and leaders enable them. We work on the cultural, organization, and process level to enable agility in the company.

Workshops with key managers will

  • Discuss the fact that the standard competencies of agile leadership and coaching – mentoring, facilitating, problem-solving, and conflict navigating – are essential to agile adoption, but simply not enough to sustain and grow agility within an organization. We find that the unique blend of combining senior internal leaders with experienced agile coaches provides an unprecedented learning environment for all.
  • Understand that most organizations adopt agile "Outside-In" - that means they start with a process change and expose organizational impediments. The problem is that most of the organizational impediments are driven from cultural values deep within the organization. the process changes rarely stick,
  • Identify an "Inside-Out" approach - that means we will start with the company culture. Just as understanding the personality of a person is a key to working effectively with that person, understanding the culture of an organization is critical to its success in adopting, and more importantly sustaining, agility.
  • Explore a number of organizational systems and structures which enable (or impede) agility at an organizational level. This approach sets the priority first to the culture, second to the structure and third to the processes of the company

We recommend

  1. Workshop with key managers as described above,
  2. One Coach at C-level,
  3. Transform the company one product line after the other, delay the broad deployment to have time to learn and improve,
  4. Study LeSS as a valuable approach. We strongly support LeSS as an ideal candidate for company wide agile approach,
  5. Implement selected company-wide measures during the second phase of the change management process when the development department moves to proficient level. Top managers must realize that cultural elements must first be modified before the structure and the processes of the company can be adapted,
  6. Be patient and do not loose your mojo.

Checklist

Here a checklist example to reflect why and how to transform into an agile organization

  • Answer the question, “Why move to Agile?” This is important, as the reasons for attempting such a fundamental change should be well understood from both a quantitative as well as qualitative standpoint,
  • Understand the current business culture. Change is hard and there will be champions as well as potential saboteurs of the changes to come,
  • Spend time on the organization structure to understand how it helps or hinders the move to agile and lean,
  • Involve all levels of the business, including top level ‘C’ executives. Their sponsorship and support will be important,
  • Don’t neglect mid-level management as their support is vital to the success of the transformation,
  • Create a roadmap with the explicit understanding that it will change over time,
  • Don’t attempt to change everything. Pick an area where a win will be evident and beneficial,
  • Prefer narrow and deep change to broad and shallow approach.

What does an Agile Coach?

What does an Agile Coach?

2018 08 01 head

An agile coach has three main areas of activity:

Foster Autonomy
Teach and support the team and individuals to be autonomous within the given boundaries of the organization and its processes.
Create Ground for Motivation
Help teams to maximize their autonomy, individual mastery and make sure the team has a purpose.
Provide Coaching
Support teams and individuals to identify potentials with their roles and within the agile organization and improve.

How Can You Do It?

Lateral Leadership
Mediating the team through conflicts and help them with making decisions and foster the team’s self-organization. After a few months a team shall have jelled.
Facilitation
Facilitate meetings for the team, including preparing, moderation and post processing for the agile process as well as the supporting agile organization. After a year of working with a team, the team members shall be empowered to run regularly all events without you attending.
Teaching
Teach teams agile practices and lateral leadership. A central area is technical excellence and software craftsmanship. Long lasting teaching shall concentrate on values: Scrum values and principles, Less principles, Agile manifesto and the associated 12 principles.
Impediments
Actively identify impediments and foster their removal. Consult the Scrum guide: You must select one impediment after each retrospective and add it as high priority item in the next Sprint backlog. This mechanism insure continuous improvement.

To ensure continuous improvement, it includes at least one high priority way in which the team works, identified in the previous Retrospective meeting.

Culture
Establish a culture of inspect and adapt for teams , create structure to establish a continuous improvement process - also meaning establish a tolerance to failure -, advocate transparency, enlighten the organization of the relation between autonomy and accountability.

Ask yourself how your organisation handles meritocracy.

One Approach to Coaching

Agile teams thrive with purpose, autonomy and mastery. One approach to coaching is to set your focus on

  1. Understand the domain: Talk with customers and users and understand their needs. Your teams should have business expertise,
  2. Master Technical Excellence: CI, CD, TDD, ATDD, pair programming, refactoring, lean UX, usability
  3. Use Process Excellence and apply Scrum, XP, LeSS by the book:
    1. Discuss the core values of commitment, focus, openness, respect, and courage,
    2. Present again the agile manifesto and the twelve principles
    3. Show the organization why commitment in the planning was replaced by forecast in the official Scrum guide,
    4. Require a vision, a roadmap, a release plan with a story map and a refined backlog for each product and team,
    5. Tell why an impediment resolution backlog item shall be part of each iteration,
    6. Teach the importance of Definition of Done for the internal quality of the product. And explain why Definition of Ready is often an anti-pattern,
    7. Show why velocity is not a productivity metrics,
    8. Show the difference between product and output,
    9. Extend the agile toolbox with Design Thinking, Lean Factory, Lean UX, DevOps,
    10. Pair Scrum master and technical master to implement a craftsmanship path and achieve product quality,
    11. Explore Center of Competencies to nurture your product development group,

Once you are a master fell free to break the rules, experiment with new rules, approaches. Define hypotheses and verify them empirically.

If you are coaching software development groups acknowledge you shall possess profound understanding of technical excellence and software development practices. I never heard of a soccer coach not intimately knowing how the game is played.

Architectural Decision Records ADR

Architectural Decision Records ADR

2018 07 01 head

Architecture Decision Record ADR is a compact and traceable documentation of all important decisions taken during the realization of a product. Use asciidoc or Markdown and Git to have an agile and lean documentation of your major decisions.

The huge advantage of Architecture Design Record ADR is the timeline of the decisions. It tremendously helps to understand the technical changes during the product lifecycle.

Definition

In each ADR file, write these sections:

Title
short present tense imperative phrase, less than 50 characters, like a git commit message.
Status
proposed, accepted, rejected, deprecated, superseded
Context
what is the issue that we’re seeing that is motivating this decision or change.
Decision
what is the change that we’re actually proposing or doing.
Consequences
what becomes easier or more difficult to do because of this change.

I use asciidoc format to write all our architecture design records. You can add to the same git repository as the product, visualize differences and if desired generate a nice looking document. And my IDE IntelliJ IDEA has a nice plugin for this format.

Before using ADR I documented the decision changes as footnote in a word document. The ADR approach is way more legible and easier to trace with the help of git repositories.

Title

These documents have names that are short noun phrases. For example, "ADR 1: Deployment on Ruby on Rails 3.0.10" or "ADR 9: LDAP for Multiple tenant Integration"

Context

This section describes the forces at play, including technological, political, social, and product local. These forces are probably in tension, and should be called out as such. The language in this section is value-neutral. It is simply describing facts.

Decision

This section describes our response to these forces. It is stated in full sentences, with active voice. "We will …​"

Status

A decision may be "proposed" if the product stakeholders haven’t agreed with it yet, or "accepted" once it is agreed. If a later ADR changes or reverses a decision, it may be marked as "deprecated" or "superseded" with a reference to its replacement(s).

The status and the graph of related decisions can be tracked and visualize using tools. You can also track your decisions without any tools, a regular product has often less than hundred decisions. And you can always write a small parse utility to extract your dependency graph.

Consequences

This section describes the resulting context, after applying the decision. All consequences should be listed here, not just the "positive" ones. A particular decision may have positive, negative, and neutral consequences, but all of them affect the team and product in the future.

Tips and Tricks

The whole document describing an ADR should be one or two pages long. We will write each ADR as if it is a conversation with a future developer. This requires good writing style, with full sentences organized into paragraphs. Bullets are acceptable only for visual style, not as an excuse for writing sentence fragments (Bullets kill people, even PowerPoint bullets).

The approach is very similar to how you would document a software pattern.

Believe and respect is the prime directive.

Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.

Norman Kerth

How to reach the Software Quality Graal?

2018 06 01 head

Lately I was asked how many bugs and smells detected through SonarQube were acceptable for a software product. And a Scrum master inquired which organizational changes are necessary to empower software developers to write good code. Speechless and dumbfounded I was unable to answer these questions.

After quite some time I realized these persons believe tools can measure the quality of software. But tools can only measure non-quality.

They tell you when you are doing it wrong, only your heart can tell you when you are doing it right.

But first that does it means to have a quality software product?

A high quality software product has no catastrophic failures, and seldom bad behavior.

It is maintainable and extendable with new technologies and new features.

The first part is called extrinsic quality and is why customers buy your product. The second part is called intrinsic quality and is why customers buy your product again and recommend it.

Criteria such as test code coverage, static analyzers, coding guidelines are just tools to detect missing quality. They help your team produce higher quality and nothing else.

I encounter organizations stating that zero defects as measured with SonarQube is the ultimate goal - and they find normal to have hundred of thousands of smells -. Time for a reality check.

Tools such as SonarQube, unit test coverage are just instruments to support your teams to detect known non-quality.

How to find the Quality Graal?

Below I provide a set of established, old techniques to eliminate non-quality. Most of them are around for at least twenty years. I encourage you to extend your practices with additional procedures based on your measurements and findings.

Extrinsic Quality - Product Quality

Is why you buy a specific car

  • Acceptance criteria and behavior test driven development ATDD - You insure through automatic tests the feature behave how your customer requested,
  • Defect driven development - You insure through automatic tests your customer will never encounter this exact error again,
  • Fast release and deployment cycle
    • You give your customer a way to detect bad behavior early and have an opportunity to ask for improvement. By fast we mean daily or perhaps weekly.

Intrinsic Quality - Technical Quality

Is why you can daily drive your car and not have to weekly repair it. And, later be able to add a charger for your mobile phone without spending a fortune.

  • Static code analysis - Known weaknesses are removed from your source code,
  • Test driven development TDD - You can change the code without changing the behavior or outputs, see also ATDD,
  • Clean code - You guarantee the source code is maintainable,
  • Clean architecture and domain driven development - You guarantee the product is extendable.

Accountability

The Scrum Team - Development Team, Product Owner and Scrum Master - are responsible and accountable for the intrinsic and extrinsic quality of the delivered products. Insufficient quality is always a collective failure.

Truly if the intrinsic quality of the source code is low, it is undeniably a failure. A failure of the development team and of each software developer writing the code; and a failure of the Scrum master holding and teaching agile and Scrum values.

When there is high pressure on delivery people have a tendency to pay less attention to processes and standards. That is one of the reasons why in Scrum we have the role of Scrum master responsible for understanding and enacting agile practices and rules.

And no physician will ever cut corners just because he is in a stressful situation. So please be a professional and read again the principles of the agile manifesto such as

Continuous attention to technical excellence and good design enhances agility.

Agile Manifesto

Developers please read the seminal book Clean Coder [2] from Robert Martin. As a professional you are responsible and accountable for the quality of the software you produce.

I find intolerable to hear software professionals stating they are not in charge of the intrinsic quality of the product they write. They cowardly try to push the responsibility to the internal quality assurance department or to the whole organization.

The above ideas are keys for Software Craftsmanship.

Kaizen - Small Steps bring First Successes

The first steps to improve your product are quite simple are around for at least twenty years.

  • Eliminate all compiler errors and warnings,
  • Use static checkers,
  • Activate the checkers of the IDE you are working with. For example, I am using IntelliJ IDEA and they have extensive checkers,
  • If using SonarQube, download {url-articles}/2018/blog-2018-05-01]SonarLint plugin and simply use it -. This tool is best of breed, do not argument and just use it -,
  • Write tests with JUnit5, Mockito, AssertJ, Cucumber.

I am ashamed to see software developers not implementing the above measures on a daily basis. I despair encountering Scrum masters who tolerate the situation. These techniques cost nothing, are installed in a few minutes and support incremental improvement of your source code. See also Why software developers do not need to ask for permission or forgiveness.

Your goal is simply zero warnings at the end of the spring. Later tighten it to zero warnings in the trunk of your git repository. Just do it - and stop arguing about the merits and costs -!

And if you do not agree with one of the warnings just disable the rule and document the reason in an architecture design record ADR.

Agile and Scrum are all about learning and continuous improvement. Please stop talking about quality and just start improving it! I do not want to hear any lame excuses why a software developer is not continuously doing the above activities.

I acknowledge I am emotionally involved and not fully objective. Lately I saw Java code of three different commercial products and I am still recovering from the missing quality.

Literature

  • [1] Clean Code, Robert C. Martin, 2009
  • [2] Clean Coder, Robert C. Martin, 2011
  • [3] Clean Architecture, Robert C. Martin, 2018
  • [4] Clean Agile, Robert C. Martin, 2020
  • [5] The pragmatic Programmer: Your Journey to Mastery (20th Anniversary Edition), Andrew Hunt & David Thomas, 2019

Agile Trends Switzerland 2018

2018 05 02 head

What are the main hurdles to introduce agile approaches in Swiss companies. The Swiss agile trends report was published for the last ten years and learn us the difficulties and successes of agile transformations.

SwissQ has published a "SwissQ Software Development Switzerland 2018", the tenth version of this report.

Introducing agile company-wide was and is a cultural change process. Such a change takes time and sometimes hurts. Too often organizations stop the process and revert to all command and control approaches with Waterfall product management.

Without commitment of senior management, the initiative will fail. In other words Walk the talk.

You must teach, coach, and support your collaborators through the transition. You need to believe in them and provide an environment where they can safely learn and experiment.

At the end what matters is collaborator purpose, customer satisfaction, and business value.

The study can be downloaded as PDF from their web site. See previous blogs for the results of years 2016, 2013 and 2012. Below some of the findings of the study.

Slower Adoption of Agile Approaches

The adoption of agile frameworks has markedly slowed down in Switzerland. And often hybrid methods based on Waterfall methodology are used. It is fascinating to see companies trying to merge a linear approach with highly iterative and incremental approaches. It shall be clear that success will only be elusive.

The data of the study reflect this fact through the abysmal collaborator satisfaction with these hybrid approaches (see chapter 13). Less than 25% of employees are satisfied with the development process of their company.

Major Hurdles When Introducing Agile Approaches

The list of hurdles for a successful introduction of agile approaches was updated.

69%
Culture change and you need senior management support
54%
Non adapted overall processes because local optimizations have limited effect
42%
Missing product overview because a company sells products
42%
Focus on business value because a company shall earn money
25%
Missing agile know-how of collaborators and you shall train your collaborators

We reached a major threshold. The majority of interviewed companies have more than 500 IT collaborators. These big companies are struggling to introduce agile approaches company-wide. Interestingly 75% of all companies plan to increase investment in the agile introduction and transition projects.

One interesting consequence is the company compiling the report is actively poaching potential agile candidates from these big laggard companies!

The majority of teams use Scrum or Kanban. The associated events and tools are used. The deeper values such as test driven development, acceptance test driven development, clean code or refactoring are often missing. The key is:

To be agile, not to do agile

We wrote in previous blogs that introducing Scrum in the development department or in the whole company is a change process. The coaches should be trained in change management and have experiences with resistance to changes. Our experience shows senior management are often not ready to move along the cultural change path.

Sonar Lint for the Impatient

Sonar Lint for the Impatient

2018 05 01 head

Remove issues before they reach customer sites. No quality problems shall reach your Git master branch.

What is Sonar?

SonarQube is a central server that processes full analyses (triggered by the various SonarQube Scanners). Its purpose is to give a 360° vision of the quality of your code base. For this, it analyzes all the source lines of your product on a regular basis.

It should be added that SonarQube also performs scans with well-known third party analyzers (SpotBugs - previously FindBugs -, Checkstyle, PMD) whereas SonarLint does not include those. I think the reason is a prioritization on performance and findBugs relying on java byte-code.

What is SonarLint?

SonarLint lives only in your IDE (IntelliJ, Eclipse or Visual Studio). Its purpose is to give instantaneous feedback as you type your code. For this, it concentrates on what code you are adding or updating.

Findings are displayed in the source code with information such as rule violation, rating of the rule and explanation. You have a window where all findings are displayed to provide an overview and navigation support. You are free to analyze a single file, a package, or the whole product; you can also exclude files or packages from the analysis.

Connect SonarLint to your preferred code quality application. It then uses the same code analyzers, rules and settings locally that are used on the server.

Issues marked as Won’t Fix or False Positive in SonarQube or SonarCloud disappear from your IDE.

Both SonarLint and SonarQube rely on the same static source code analyzers - most of them being written using SonarSource technology.

How to use It?

  1. the SonarLint in your favorite IDE
    1. configure the SonarCube instance to synchronize the rule set used in your product
  2. Configure the plugin to access your SonarCube instance
  3. Run it,
    1. See the messages in the next few seconds
    2. Jump to the source and correct the problem
    3. Iterate until all issues are solved
  4. Run your unit tests and acceptance tests - TDD and ATDD -
  5. Commit your changes

This approach guarantees you always release source code with less SonarCube issues, therefore SonarCube always show no findings for new code and for refactored code.

And you have made a step to the journey to become a professional software developer and achieve {url-articles}2018/blog-2018-04-01[craftsmanship].

Agile Introduction Success Criteria through the LeSS lens

Agile Introduction Success Criteria through the LeSS lens

2018 04 02 head

An agile organization is able to deliver or change direction at any time without additional cost.

More companies want to become truly agile and book an expensive journey to the holy land of agility. Often the travelers never reach their true destination.

The LeSS people have a clear set of conditions to increase the odds.

All my experiences confirm these conditions are essential for success.

LeSS Criteria

Use these criteria as a checklist to evaluate your current agile transformation

  • Educate everyone
    • Train all involved collaborators extensively, train them multiple times,
    • let them apply the learnt to their daily activities,
    • Select an experienced coach understanding organization, team, and technical excellence,
    • Plan for a long term engagement of the coach. The journey will take years, *Define your “product”
    • You need an elevator pitch describing your product family and why the customers is buying,
    • Identify your vision, your roadmap for the next three years, your releases for the next 12 months, and product backlog and boundaries of your product,
  • Define “done” - Definition of Done
    • Definition of done tells when your work is completed,
    • A strong definition of done supports organizational changes, see the LeSS metaphor of the Undone Department,
    • Check the team can implement the current definition of done or expand cross-functionality and training,
  • Have appropriately structured teams
    • A team is stable, long-lived, colocated, cross-functional and implement end to end features for the solution. Avoid component teams specialized for database, user interface or communication layer,
  • Team members are dedicated to the team. They do not work in other projects and do not report to a matrix structure,
  • Only the Product Owner provides work for the teams
    • The team is dedicated to implement the best solutions as defined by the product owner. No side activities are tolerated,
  • Keep product managers away from the teams
    • Project managers are not needed in an agile organization,
    • We realize and sells products, projects are irrelevant,
    • Nurture good managers to sustain the transformation.

Further Reading Read the LeSS Rules for ideas and wisdom how to improve your approach to agile product development.

As you found out now, I am convinced that LeSS approach is one of the best ways to truly transform into an agile organization. And less truly and deeply advocate technical excellence for all collaborators. You need an agile mindset and technical excellence to develop awesome products - see Software Craftsmanship blog.

A huge advantage of LeSS is that it is Scrum. You do not need to learn new methodologies, terminology or processes. Just apply all your hard earned learnings from your Scrum transformation.

Pragmatic Craftsmanship - Professional Software Developer

Pragmatic Craftsmanship - Professional Software Developer

2018 04 01 head

The last months I was often confronted with software products having insufficient quality.

Insufficient means late delivery, and few new features. The effort to correct errors in the field eats 30% to 50% away from the whole development budget. How can such a dreadful situation occur?

One root cause is the low quality of the source code and ignorance of basic coding design approaches. We are not talking about complex design activities or complex patterns; we are discussing basic approaches how to write correct and maintainable code.

Master Your Technology Stack

Most of the programmers have no formal training in the technology stack they are using daily. Their best friend is the Internet. And therefore many agile projects are now, steadily and iteratively producing mediocre software.

You shall be a proud software craftsman. You shall work as a professional and deliver professional results. I recommend for any Java developer

  • Have formal training in the Java language and library. For example you should consider a Java Programmer certification for the current JDK you are working daily with,
  • Learn the new features in Java e.g. Streams, Lambdas, Reactive Programming, LocalDate, Modules, packaging with jlink,
  • Read and understand Effective Java <<[effectivejava>> from Joshua Bloch,
  • Read and understand Clean Code [1], Clean Coder [2], Clean Architecture [3] and Clean Agile [4],
  • Read and understand Refactoring [9] from Martin Fowler,
  • And read Refactoring not on the backlog article from Ron Jeffries
  • Learn modern practices such as logging with slf4j, TDD with JUnit5, Mockito and AssertJ, ATDD with Cucumber,
  • Lean DevOps, continuous integration, continuous delivery, continuous deployment, monitoring,
  • Be aware of famous Java libraries such as Guava, Apache Commons,
  • Use SonarQube - and the SonarLint plugin for your preferred IDE - to catch well-known weaknesses in your source code and improve your coding skills,

A software developer not knowing the above items is almost certainly not a professional person, just a more or less talented amateur. See the blog of Mike Cohn the difference between aprofessional and an amateur for a similar point of view.

Improve Continuously You shall strive for mastery.

The feeling when you reached a certain mastery level is truly awesome. * Read a technical book each quarter, * Lean a new language or framework every two years, * Read books such as The pragmatic Programmer [5] * Know approaches such eXtreme Programming, and read the book <<[extremeprogramming>> * Work with Scrum, Kanban, LeSS -. Agile approaches are definitively state of the industry techniques -, * Master container approaches with Docker - containers are here to stay and will replace virtual machines -.

You shall write good enough software without errors and using best practices of the industry. If not consider changing your profession. See my blog post about technical debt discussing the consequences of missing mastery.

Software Craftsmanship

Software craftsmanship is an initiative to improve the professionalism of software developers. You do not have to agree with them, just look what they are doing.

For a detailed discussion read for example the book The Software Craftsman: Professionalism, Pragmatism, Pride by Sandro Mancuso.

Don’t discuss practices, discuss value

Their manifesto is

Craftsmanship is not enough to guarantee the success of a product but the lack of it can be the main cause of its failure.

Agile and Craftsmanship complement each other and both are necessary

Agile processes assumes technical excellence and a professional attitude

Software craftsmanship takes technical excellence and professionalism to a whole new level

Take everything you read with a grain of salt. Software craftsmanship

  • Is not a church, trying to convert all developers,
  • Is about leading by example and showing how we can be better,
  • Is about well written and designed code - and the customer stays in the center -,
  • Is about continuously delivering value not writing crap code.

Your goal shall to become a professional developer and reach mastery. How you do it is up to you.

Literature

  • [1] Clean Code, Robert C. Martin, 2009
  • [2] Clean Coder, Robert C. Martin, 2011
  • [3] Clean Architecture, Robert C. Martin, 2018
  • [4] Clean Agile, Robert C. Martin, 2020
  • [5] The pragmatic Programmer: Your Journey to Mastery (20th Anniversary Edition), Andrew Hunt & David Thomas, 2019
  • [6] eXtreme Programming Explained: Embrace Change, Ken Beck, 2005
  • [7] The Software Craftsman: Professionalism Pragmatism Pride, Sandro Mancuso, 2015
  • [8] Effective Java: Best Practices for the Java Platform (3rd Edition), Joshua Bloch, 2018
  • [9] Refactoring: Improving the Design of Existing Code (2nd Edition), Martin Fowler, 2018

An extensive list of books can be found under ../../ideas/learnings/books[Books].

Why Do You Need Metrics and KPI in Agile Product Development?

Why Do You Need Metrics and KPI in Agile Product Development?

2018 03 03 head

Why do we define metrics and KPI?

First we want to find out how good the build-in quality of our software artifacts is. Second we want to improve the build-in quality through experiments and use the metrics to judge success of these experiments.

And the Key Performance Indicators - KPI - provide plausibility checks by qualifying the intrinsic quality of software artifacts against quantifiable strategic objectives.

Measurements shall be cheap to gather and as much as possible generated automatically through the tooling landscape.

We provide two sets of measurements to find out how supportive the development process is for quality and how to evaluate the quality of the artifacts.

Definitions

A metric is a measure or a combination of measures for quantitatively assessing or improving a product, a process or a team.

A key performance indicator is a metric that

  • is tied to a strategic objective,
  • have at least one defined time-bound target value - a number, a trend, a variation, etc. -

Therefore you must defined your strategic objectives before you can use KPIs.

Lean approaches have an imperative to reduce cycle-time, eliminate waste and tend to zero-defect products. The last one is a corollary because defects are waste. Please use this imperative to define your agile and lean KPIs.

If your tooling landscape does not support automatic gathering of a specific metric, you shall make it optional until you have solved your tooling weaknesses. Accurate and real-time automated measurements are of tremendous importance for acceptance and lasting success.

Process Aspects

The optimal lean software development process has a batch size of one, almost no waiting time, no superfluous stock and no quality issues.

Cycle Time of defects
It shows the effectivity to resolve non-quality in product development,
Cumulative Flow Diagram for defects and for stories/Backlog Items
It shows how effectively resources are allocated,
Sprint and PI burndown planned stories and effort
versus completed stories and effective effort trends show the stability and adaptability of the development process,
Planned velocity and effective velocity trends
It is an indicator for the maturity of the agile product organization.

This metrics shall be automatically provided through the issue and product backlog system. For example the “best of breed” JIRA tool has build-in reports for all the above metrics, so the tools you use should also have them.

Technical quality

A high quality software artifact has no code smell issues, no compiler warnings, always compile, and all tests are successfully processed.

  • Code quality
    • Method length distribution,
    • Class size distribution,
    • Cyclomatic complexity distribution,
    • Code Smells,
  • Product quality
    • continuous integration successful build trends,
    • test coverage trends for unit tests, integration tests and -ility tests. This metric implies the development team uses unit testing, TDD, ATDD, and automatic -ility testing,
  • Architecture
    • Separation of API and implementation - can only be inferred on modern technology stacks -,
  • Work techniques
    • Refactoring trends - sometimes needs to be evaluated manually in some projects -,
    • Emergent Architecture documentation trends - probably needs to be evaluated manually in most projects.

These metrics shall be automatically provided through your quality tracking and continuous integration systems. For example the "best of breed" tools SonarQube and Jenkins have build-in reports for these metrics, so the tools you use should also have them.

Remainder

The above metrics are tools to discover discrepancies and weaknesses in your product development process and generated artifacts. The inferred results shall be used to identify and implement corrective measures.

Agile emphasize transparency. The above metrics and KPIs shall be visible anytime to all involved participants through dashboards.

Agile GIS Open Data Initiative

2018 03 02 head

How can agile approaches be applied to deliver solutions in a government environment?

Here an example of a GIS open data portal successfully developed using agile approaches such as Kanban, lean product and technical excellence.

GIS Open Data Initiative

Swiss government has initiated an open government data strategy to provide data at federal and state level to all citizens and interested parties. The aims are

Release of official data
The Confederation will make data which is suitable for OGD available to the public in machine-readable and open formats to be freely reused. The individual administrative units must review and, where necessary, adapt the legal, organizational, financial and technical framework of data production to make this possible.
Coordinated publication and provision of official data
With the help of a central infrastructure (OGD portal), the Confederation will make its data sets easily locatable for the public and will make the data available for free use.
Establishing an open-data culture
The Confederation will support the use of data through free, uniform and understandable terms of use and through additional information on the individual data sets. It will also pursue a continuous dialogue with the public.

You can find more details under Open Government Data Switzerland.

The solution realized with agile approaches is a GIS open data portal supporting self-service and providing over hundred theme maps with GIS data in multiple formats, resolutions and geographical sizes available through a self-service portal.

Agile Approach

The solution was implemented during year 2017 and released to public end of year.

We identified these key elements to explain the success of the whole product development. The whole organization and all involved managers shall

  • trust the team will solve adequately and pragmatically the customer needs. Trust your team and create a work environment to support them. For example you should
    • Promote technical excellence,
    • Provide approaches for face 2 face communication,
    • Publish Kanban board to increase transparency,
  • Act as an agile leader
    • Encourage accountability and self-management,
    • Delegate and truly support technical autonomy,
  • Be a gatekeeper to support and protect your team
    • Reduce tension with IT, legal and management systems and build bridges to these departments,
    • Set reasonable and achievable goals.

The delivered solution was

  • Integrated in existing GIS portal of Zürich state to minimize barrier to interested parties,
  • Self-service approach to minimize running costs,
  • Private cloud approach due to regulatory aspects,
  • REST service to provide human readable and machine readable maps in various formats,
  • Build on existing approaches to generate GIS data to minimize risks and respect ambitious timeline.

A description of the technical solution in German can be found here. The REST API is described here.

Learnings

Pragmatic approaches to become agile were very successful in this setup. These governmental organizations were willing and eager to try them.

Senior management was supportive and after initial doubts the team embraced the approach. They enjoyed the whole process and the successful deployment of the solution.

The enablers were * Senior management committed to the approach and supported it, * Team was open to new ways of working, * Team members were competent.

Java 10 aka 18.3 Changes

Java 10 aka 18.3 Changes

2018 03 01 head

For Java to remain competitive it must not just continue to move forward — it must move forward faster.

Mark Reinhold

What has changed?

Oracle has decided to release a new JDK two times a year.

The next release Java 10 aka 18.3 will be available on of March 2018.

JDK 11 aka 18.9 will be released in September 2018. Java 11 will also be a long term support LTS release.

The Visible Change

The visible change is the dawn of the keyword var. Now instead of typing the type of a local variable - local to a method, a block or an instruction - you can just use the keyword var. The constraint being that the variable is initialized at declaration. Stay tune for more support in JDK 11.

When the type declaration is a complex generic type definition the var shortcut is a boon. The code is less verbose and the legibility is at least as good. When the type is a class name I got the impression the benefit is less clear.

As an exercise I updated the source code of open source libraries I have written and the associated tests as in

Event <Events> event = new Event<>(Events.AA_AB);
var event = new Event<>(Events.AA_AB);

The first impression is that the new construct sometimes simplify the code of the library and very often improve the test cases. Perhaps it takes time until I am accustomed to the new way of writing Java code.

Under the Hood Changes

The source code of the JDK is consolidated in one repository. It makes development work more efficient.

The garbage collector shall have a clear and documented API. Developing and adding new garbage collectors is easier.

Various updates such as Unicode newest version. The extensions are used in library classes to better support locales.

Open JDK and Oracle JDK are slowly merging. For example CA certificates will be provided to OpenJDK by Oracle.

Outlook

The rhythm of releases is staggering higher. Starting Spring 2018 we will have * The new rule as blogged by Mark Reinhold Moving Java Forward Faster. * You will get a new JDK every six months * You will get security updates every three months * You will get a long term support release every three years.

This means that you shall update your JDK in development environment six times a year if you want to install the new JDK and the security releases. We recommend automating the process in your environment. And you shall deliver your applications in Docker containers to support the whole ecosystem.

The tool landscape is still struggling to adapt to the new rhythm. I am working with IntelliJ IDE and I had to wait for the EAP 2018.3 before being able to write JDK 10 code in the IDE. I use Gradle as build tool and also to wait until JDK 10 was at least partially supported. And I am still waiting for the update of code coverage plugin. Before that I was back to stone age, writing software with a text editor and compiling it with the command line.

The library landscape is often behind and regularly they are incompatible with the newest version of the JDK.

JavaFX will be removed from the regular JDK distribution in Fall 2018.

The cloud landscape does not support latest JDK as SAAS. You must provide your custom docker images with the expected JDK or JRE. So we are back to IAAS.

Six Levers for better Agile Product Development

Six Levers for better Agile Product Development

2018 02 02 head

How can you create better products and services?

How can you delight your customers, your collaborators, and your stakeholders?

We identified six levers to multiply the effectivity of your agile development teams and empower them to create great services and products.

Please add these tools to your toolbox and create great products.

Design Thinking and Customer Experience

You solve customer problems through intuitive services and products. Customers' expectations should fit with their experience with your products. That is why they should be involved in the whole definition and realization process. At the beginning, to understand the need and to test first ideas. During the implementation, they are directly asked for feedback, which then flows into the further development steps.

Go out with your incomplete product as soon as possible. Each iteration should deliver an upgraded version of the product. The pressure to deliver is huge, a design thinking iteration has a duration of at most one week.

Lean Startup and Lean UX

Minimum viable products - MVP - are first developed as minimally functional products. Within 60 days, a physical prototype will be created that customers can test during this time. The software component shall be delivered after the first or second iteration, that means in less than one month.

New technologies such as 3D printing, cheap hardware and IoT boards, continuous delivery, simulation enable your teams to deliver on time. These are not pure mockups but executable applications. If it appeals to the customer, the MVP will evolve gradually to a minimum marketable product - MMP -. The focus is always on adding customer value.

Beyond Budgeting Approaches

There is no long-term budget plan for products and projects. After each cycle of customer feedback the goals and budget are evaluated again. The value streams have agile teams that work like small companies.

Each team has a Product Owner - PO - who has overall responsibility for their respective product. Realizations are further funded, modified or canceled on the basis of initial results, meaning after customer and technical tests.

The product financing is following a start-up approach and not based on yearly budget as usual in larger companies.

Co-location and Team ownership

Work is done across departments, in cross-functional teams. All those who can contribute to the success of the product (e.g. company organization, IT, market management, or sales) sit in one room. They form the agile team. Co-location ensures lean coordination processes as well as speed and quality.

Co-location also means: You are moving. The teams move into a room and work focused together on their product. There are no phones, no other projects, no outside distractions. They use collaboration tools - chats, wikis, Scrum boards -. Scrum teams are wholly focused on their product,

People checks three times a day their emails; otherwise the email client is closed and inactive. All phones have voicemail.

Agile Methods and Craftsmanship

Employee training is essential. In the value streams, we provide staff with experienced coaches to build knowledge in agile methods and software craftsmanship.

Work is done according to proven agile methods. Software development relies on modern and agile techniques such as pair programming, refactoring, test driven development, continuous integration, and Scrum. For bigger product development we use LeSS approach.

Read this {url-articles}2018/blog-2018-01-01[blog] for a description of modern agile development procedures.

Elastic Infrastructure

We shall use state of the industry technology. This enables test-driven development - ATDD, TDD -, meaning predefined automated software tests that run in real-time.

Cloud infrastructure provides unlimited processing capabilities for development, test, and deployment of software artifacts.

Developers have fast notebooks, fast SSD, plenty of RAM, and 4K external screens. In addition, the technical infrastructure enables software to be scaled and adjustments made immediately - CI, CD, DevOps -. The goal is to deliver a software release in production every few hours if needed.

Learnt Lessons

  1. The power of co-location is tremendous. The work atmosphere is focused, friendly and productive. Team members are continuously working on the solution without spurious task switches in their team room,
  2. Establish stable and long living teams,
  3. Realize the importance of training and coaching,
  4. Embed your teams and processes in the organization,
  5. Change processes for beyond budgeting approaches and lean enterprise. The organization has to stop requiring exact budgets and detailed yearly planning,
  6. Reflect on the impact on the whole organization. See also my blog discussing what agile companies are.

Scrum Guide 2017: What changed?

Scrum Guide 2017: What changed?

2018 02 01 head

The Scrum guide was revised in 2016 and got clarification concerning the core values. I was thrilled. Talking about values and mental models you must have to successfully introduce and be agile is awesome.

A new revision was published end of year 2017, just a year after the values addition.

What has changed? What have the community learnt and committed to the new version of the guide.

Scrum is not only for Software

Scrum was initially developed for managing and developing products. Starting in the early 1990s, Scrum has been used extensively, worldwide, to:

  1. Research and identify viable markets, technologies, and product capabilities,
  2. Develop products and enhancements,
  3. Release products and enhancements, as frequently as many times per day,
  4. Develop and sustain Cloud (online, secure, on-demand) and other operational environments for application use,
  5. Sustain and renew products.

The essence of Scrum is a small team of people. The individual team is highly flexible and adaptive. These strengths continue operating in single, several, and networks of teams. They develop, release, operate and sustain the work and work products of thousands of people.

They collaborate and interoperate through sophisticated development architectures and target release environments.

Scrum is a tool to solve complex problems, any kind of complex problem. It is already the industry standard for software projects, it is becoming a major standard for all complex products.

What does a Scrum Master?

Changed wording in The Scrum Master section to provide better clarity to the role. The text now reads:

The Scrum Master is responsible for promoting and supporting Scrum as defined in the Scrum Guide. Scrum Masters do this by helping everyone understand Scrum theory, practices, rules, and values.

The Scrum Master is a servant-leader for the Scrum Team. The Scrum Master helps those outside the Scrum Team understand which of their interactions with the Scrum Team are helpful and which are not.

The Scrum Master helps everyone change these interactions to maximize the value created by the Scrum Team.

Added to the section Scrum Master Service to the Product Owner

Ensuring that goals, scope, and product domain are understood by everyone on the Scrum Team as well as possible.

Scrum master is a servant leader and promotes Scrum and agile approaches. The whole team work on a product. They understand the vision, goals, scope, and have extensive domain knowledge.

This means the whole team talks with stakeholders and users. The team members have a genuine interest to learn more about the product domain.

Scrum Events

Updated the first paragraph of the Daily Scrum section to read
The Daily Scrum is a 15-minute time-boxed event for the Development Team. The Daily Scrum is held every day of the Sprint. At it, the Development Team plans work for the next 24 hours. This optimizes team collaboration and performance by inspecting the work since the last Daily Scrum and forecasting upcoming Sprint work. The Daily Scrum is held at the same time and place each day to reduce complexity.
Updated the Daily Scrum section to provide clarity on the goals of the Daily Scrum including this text
The structure of the meeting is set by the Development Team and can be conducted in different ways if it focuses on progress toward the Sprint Goal. Some Development Teams will use questions, some will be more discussion based. Here is an example of what might be used:
  • What did I do yesterday that helped the Development Team meet the Sprint Goal?
  • What will I do today to help the Development Team meet the Sprint Goal?
  • Do I see any impediment that prevents me or the Development Team from meeting the Sprint Goal?
Added clarity around time-boxes
Using the words “at most” to remove any questions that the time-box for Events means maximum length, but could be shorter.

The guide clarifies the daily Scrum event. It establishes a daily feedback loop for improvement.

For slow learners time-box means maximum duration.

Scrum Artifacts

Added to the Sprint Backlog section
To ensure continuous improvement, it includes at least one high priority way in which the team works, identified in the previous Retrospective meeting.
Added clarity to the Increment section
An increment is a body of inspectable, "Done" work that supports empiricism at the end of the Sprint. The increment is a step toward a vision or goal.

The guide emphasizes the learning and improvements process. One root of the agile manifesto is lean thinking.

For slow learners you inspect the result of the increment and put at least one improvement measure in the next sprint backlog.

Previous Changes

The biggest change for the daily work was introduced in the previous revision - read previous blog -. The word commitment is no more used in the description of the planning event. The team now provides a forecast which stories shall be realized until the end of the sprint.

This change was necessary because people - especially command and control responsible - did not read the official definition of commitment - Oxford Dictionary -.

The state or quality of being dedicated to a cause, an activity.

The second big innovation was the introduction of five Scrum values: Commitment, Focus, Openness, Respect, and Courage.

I welcome the focus on values, principles and core behaviors over detailed checklists and rules. Please also read again the twelve principles of the agile manifesto.

One Way to Improve Your Scrum Approach

One Way to Improve Your Scrum Approach

2018 01 02 head

You work using agile approaches. Your collaborators are trained how to use agile approaches. Techniques such as test driven development, continuous integration or clean code are regularly practiced.

Now you are looking for the next improvement. How can your team improve product and increase customers' value?

It is time for a fresh view. Perform an agile assessment with an external expert from another company or department.

Effort

You want to take a fresh view on your process and practices. The external person shall provide hints that he observed in your team and how other groups solved similar problems. The assessment lasts approximately 2 days per team including the finding workshop with the team members, Scrum Master, and Product Owner. This workshop has around half a day duration.

Approach

The Scrum Assessment speeds up your Scrum mastery and let you collect low-hanging improvements.

  • Do you already work with Scrum or Scrum/Kanban and want to know where you stand?
  • Do you get the benefits that are possible with Scrum?
  • Where do you deviate from the Scrum theory and what are the consequences for you?
  • Are your structures and practices for coordinating dependent teams effective, meaning scale agile?
  • What potential for improvement do you have?
  • Which measures make sense so you can benefit more from Scrum?
  • Could you introduce agile techniques to increase the quality and to enhance outcomes?

We work on these themes in the Scrum Assessment.

  • We analyze your Scrum implementation,
  • We collect feedback through questionnaires and observation,
  • We assess a single team or the whole development organization,
  • We evaluate in comparison with other companies and make suggestions for improvement.

Action

We typically proceed as follow for the analysis:

  • Record Scrum meetings - sprint planning, sprint review, sprint retrospective, daily scrum, if necessary backlog refinement -,
  • Observe how the team members work during the sprint, to better understand the dynamics in your team,
  • Analyse the vision, roadmap, release plan, product backlog, and impediment list together with the product owner,
  • Interview product owners, scrum masters, selected developers and selected stakeholders,
  • Review the structures, processes and practices (e.g., Scrum of Scrums, Portfolio Kanban) used to coordinate dependent teams.

After this analysis, we do a debriefing in a half-day workshop. We present our observations, conclusions and recommendations. The Scrum team uses our findings to reinitiate their agile transformation journey.

Through the workshop approach, we tailor the recommendations together with the team members to specific needs and we can clarify any ambiguities.

Upon request, we will also prepare a written report.

Why is it so Cool to Develop with Java?

Why is it so Cool to Develop with Java?

2018 01 01 head

Why is it so cool to develop software using Java, a modern IDE and agile tools. Because the environment empowers you and incredibly speed-up your development outcomes. In seconds, you know if the next code increment adds value to your product.

Liberated your team works adding functions and no more spend unaccounted hours in setting up infrastructure.

Below our current set of techniques and tools to multiply our results and bring earlier a product to your customers.

Java Quality

Solution Quality

The most efficient approaches to improve the intrinsic quality of your software solution are

Use Pair Programming
Write regularly together source code. Decide that all complex code parts shall always be developed through pair programming. If you feel adventurous give a try to mob programming,
Use Refactoring
Before pushing your code to the Git repository, always take a look and refactor your code. You are a craftsman and shall be proud of your result,
Use Java 8 Lambdas and Streams
Actual Java code builds on lambda expressions and stream constructs. Your for loops are replaced by stream pipeline, your algorithms extensively use lambdas.
Use Java 9 Modules
It is finally possible to clearly define public interfaces of a whole software component. Configure the dependencies to other libraries, and identify specific modules as privileged participants.

Do not write Java code as it was done last millennium. Use the actual constructs to write solid and resilient code.

Code Quality

The most efficient tools to detect glitches or legacy constructs in your code are

SonarLint
The plugin for your favorite IDE provides all the SonarCube checkers to apply on your source code. Purify it before pushing it into the continuous build pipeline. The plugin support download of tailored rules from you company SonarCube instance,
SpotBugs
The successor of the wildly popular FindBugs tool can be integrated in your build pipeline to improve the odds of correctness,
PMD
Is another tool to automatically detect potential quality problems,
IntelliJ IDEA build-in Checks
You are lazy and do not want to use additional tools, please use at least the build-in checkers of IntelliJ IDEA to improve your code. It is very unprofessional to have equality problems or boolean condition errors in modern code.

Test Quality

Test Driven Development - TDD

You are serious about development quality code. So you are working with the test driven approach TDD and need to write unit tests. Each time you write some code you go through the loop "red - green - refactor - commit".

Junit 5
is the standard and best unit test framework for Java solutions. The newest version support lambdas is a pleasure to use.
AssertJ
enables the writing of legible assert statements in your unit tests.
Mockito
Tests must be lightening fast, thousands of unit tests shall run in a few seconds. Remove all dependencies to externals components from your source code, mock these elements with Mockito.
Hamcrest
Your test conditions are complexer and you want to have legible code, start using hamcrest to write clear assertions in your tests.

Please never access a database or a file system when running unit tests. If you low level tests with files just use an in-memory file system mock. [1]

Each time an error is found in productive code, write first a test to reproduce it, then correct the error. This approach guarantees the same error will never again occur.

Acceptance Test Driven Development - ATDD

You are serious about delivering the functions your customer need. Each story you implement has acceptance criteria. These acceptance criteria defines your requirements - see specification by example -. Your acceptance tests document and validate these requirements.

Cucumber
Write your acceptance criteria and document your requirements in one step.

Remember stories are not requirements, they are work items. Requirements are coded and documented in your acceptance tests.

Continuous Integration

Agile means reacting to changes. Lean means build-in quality. Your continuous pipeline certifies after each change that your solution is correct and has the correct behavior.

Gradle
Is the new standard for building your artifacts.
Jenkins
Is the regular tool for local build pipeline.

If you a choice use a cloud continuous integration for team activities. Decide if you prefer the cloud approach - e.g. with TravisCI or Bitbucket Pipelines - or Jenkins within Docker for your local integration activities.

A smooth running CI pipeline is a boost for team morale and motivation. To know your application compiles and runs through all your automated tests anytime is such a good feeling.

Continuous Delivery and Deployment

Agile means react fast the changes. You need to deliver and deploy your new solution to your customers in minutes or hours.

Docker
Virtual machines were a powerful construct but they are slow and require a lot of resources. Docker is the answer. Start a testing instance in 300 milliseconds.
Tomcat and TomEE
Embedded servers deployed in docker image in a matter of seconds. Actual versions of Tomcat or TomEE support it.
hsqldb
In memory database are lighting fast for integration and test staging areas. Later move to a file based database for the production area. The database shall be a configuration of the delivery and not require a new compilation.
Google Cloud AppEngine
To deploy various integration environments

Design Quality

Important decisions and overall structure shall be documented for new team members. Architecture Decision Record: A lightweight approach to document architecture decisions using markdown documents and traceability through the decision history. UML: Sometimes I would like to draw some UML diagrams to explain a design solution. I currently using a textual tool to define such diagrams.

Development Quality

Analog Board
Nothing beats a big analog board in the team room as the ideal information radiator. If you want your colocated to work effectively and solve problems as team, you shall provide an analog board.
Trello, Stride, BitBucket, Hipchat and others Atlassian tools for distributed teams
The first advice is to have a collocated team. The productivity and performance gains are unachievable in a distributed environment. We have very good experience with Atlassian and Google App solutions to mitigate the drawback of distributed teams and establish a team work atmosphere. But a colocated team still rocks.
IntelliJ IDEA
This IDE is the choice for refactoring, powerful context suggestions and efficient programming. If you use Eclipse or Netbeans just try a friendly refactoring competition with an IDEA convert.
Git
Please stop using Subversion, CVS, TFS or other geriatric tools. Agile development, efficient integration in delivery pipelines, and extensive refactoring require Git. Just migrate now to Git and leave dreaded message Tree conflict detected, please solve manually behind you.

These tools shall be used the agile way. You do not have long-lived branches in your source code management tool. Either you have only short-lived local branches or better you just work against trunk. Complex branching strategies are just waste for a lean practitioner.

I decided to collect all these hints in the same long post. They belong together. Every day you design a solution, you code it, test it, improve it and guarantee its quality.

These activities are tangled together to produce a product you are proud of.

And yes I know about Github, Gitlab, Slack, Ant, Maven, Jetty, Amazon Cloud, AssertJ, Subversion, Eclipse, Netbeans and other good tools. But I still prefer the ones above; I am convinced to be more productive with them.


  1. After years of neglect, a Jimfs 1.2 was released in January 2021 and published on maven central.

2017

Why Use Current Software Components?

Why Use Current Software Components?

2017 12 02 head

Often teams discuss if their product should upgrade to the current version of used components, development tools and programming languages.

I have a strong opinion about upgrades. You should always use the current version of all components included in your product. The only discussion is the timely when and never the if. Be realist, if your product is successful, you have to upgrade before the components are obsolete. Otherwise your application will become a geriatric solution.

Below I show you arguments why you should always update all components in a timely fashion.

Security and Governance

Social responsibility and liability considerations require your product to be immune to published attacks. Your company shall provide reasonable protection to all your customers. Therefore, you always must include security patches in your product in a timely manner. Other approaches are deeply unprofessional and unethical. In quite a few countries you are liable if you do not update your software to avoid known security attacks.

You should use actual Java JDK and JVM version because the community is concentrating their effort on the latest version. Oracle only provides public support for the latest two Java versions, e.g. in January 2018 Java 9 and Java 8 are officially supported. Java 8 support will stop end of September 2018, older versions are no more publicly maintained.

If for some reasons you are using an older commercially but no more publicly supported JDK, you must sign a support contract with Oracle company and pay for it. You cannot use deprecated JDK versions because nobody provides security fixes on these versions - e.g. Java 6 commercial support ended in December 2017 - .

If you include a deprecated JDK / JVM in your product, you willingly endanger your users and shall be liable for their losses.

Upgrades and Genuine Improvements

The component developers put tremendous effort to improve the non-functional aspects of their products. Performance and scalability improvements between Java versions are impressive. You shall not hinder your own customers to rip these low-hanging fruits.

Actual versions of standards - Unicode, HTTP protocol, time zones - help eliminate incompatibilities and subtle errors from use cases. Having older versions will hinder you to support actual timezone changes or extensions in the Unicode standard.

Training and Community Support

Good training and associated certification are only available for actual versions of languages, frameworks or library. You will not find any training for Ada, Apache server 1.x, or any object-oriented database.

The articles in Stack Overflow and the community always cover the current version of components. So you are on your own for support or code examples for older versions of libraries.

Hidden Cost of Backports

Writing code for older versions of libraries, frameworks, or runtimes means backporting to older API and features. For example if you do not have access to auto-closable resources you have to write your own framework and software checks guarantying closing system resources. You should always free system resources in your application. The literature states this is neither easy nor error free. It is the root reason why such a feature was added to Java.

Often teams forget the consequence of back porting. Later when you adopt a more modern version of your dependencies, you have to remove all this now spurious code and migrate to the build-in concept. Otherwise the code is no more maintainable and changes are error prone.

In other words you have to twice work when backporting, once to build it in, and a second time to later remove it. This solution is certainly not the most economical one.

Motivation of Team

No professional developer wants to write source code using JDK 7 or older versions. They want to develop using modern and powerful constructs such as lambda, modules, HTTP/2. I would state that no using such constructs disqualifies you as a professional engineer.

We observed how whole teams became demotivated because a huge part of their daily work was backporting to old environments.

I understand that geriatric software solutions do not allow you to use the latest versions. It is time to bury such products and to go on. The only open question is the date of the burial. Do not wait too long, decomposition is never nice to smell.

Maintaining or extending software based on old versions of libraries is just throwing money out the window. You need a solid basement to build on.

Clean Your Desk and Define New Year Improvements

Clean Your Desk and Define New Year Improvements

2017 12 01 head

It is Christmas time. Clean your desk and the content of your workstation. Upgrade your tools and your development environment to the actual version.

It is time to add automated build pipelines, TDD, ATDD, and Docker to your toolbox. Embrace agile approaches.

Work focused and avoid overtime. Prioritize outcomes instead of outputs. Enjoy family life, practice sport, and pursue continuous training.

Clean Your Desk

Digitalize Your Office

It is time to sell the last fax device in your office and plan to remove your big printer. Burn your paper documentation and discard your folders. Please store all information digitally and have an effective search application in your company. Use Wiki to publish company data and store knowledge for all collaborators.

Have a professional handset and a good camera for video conference.

Put all your documents on your company drive and index all this information. Finding relevant data by keyword searching is a huge gain of time. Another advantage is no more backup of workstations and notebooks. All mission critical information is always on the company’s drives.

Socialize with your Team

Organize a real team room. Colocate all your team members, design on big whiteboards. Please use analog Scrum boards. You need big whiteboards, analog Scrum boards, meeting corners.

Restart your brown bag training program. A Friday beer event is always a welcome event.

Celebrate success after each major release of your products. Another gain is a lot of tensions just disappear when people talk around snacks and drink together a glass of wine.

Upgrade your Toolbox

As a Java developer it is time to use state of the industry tools and improve your daily work

  1. Java 9 JDK - If you are still working with Java 7 or earlier you are a laggard
    1. Java 8 public support shall end middle of next year,
    2. Java 10 shall be available next year. The early releases are already online. Download them and try the new features. The local variable declarations are a huge improvement,
    3. You are a laggard and security danger if you are still using Java 6 or earlier,
  2. Gradle
    1. Build, deliver, and deploy your solution with one keystroke,
  3. Continuous integration, delivery and deployment pipelines,
    1. Never again configure manually a deployment by hand,
  4. Test environments with Docker
    1. Never say again "it is working on my machine",
  5. Modern IDE such as IntelliJ IDEA
    1. Very good Gradle support,
    2. Very good JUnit 5 support,
    3. Alt-Enter anywhere brings up context-sensitive options. Run a test, create a test, create subclass, convert to closure, generate missing test methods, remove explicit type, split into declaration and assignment, introduce local variable. The suggestions are limitless.
    4. Great Java 8 support, suggesting lambda notation where appropriate!
    5. Even displaying anonymous inner class usage (collapsed) as if it was written using a lambda expression,
  6. Quality with PMD, FindBugs, SonarLint, or the build-in rules of IntelliJ
  7. Banish Flash from your workstation

New Year Improvements

Sustainable Pace

Agile approaches state you should work at a sustainable pace, forty hours a week. Avoid working overtime during evenings or weekends.

Enjoy family time, practice sport, give back to the community.

Work with your team, ideally colocated in a team room. Face2Face exchanges and pair working - pair design, pair review, pair programming, pair checkin - are training on the job occasions.

Hone your Craftsmanship

Here some suggestions to hone your craftsmanship

  • Practice coding dojos to improve your programming skills,
  • Read good technical books, at least one book per quarter,
  • Learn new shortcuts and plugins for your favorite IDE,
  • Improve the agile, lean, Scrum, LeSS approaches in your department,
  • Refactor daily code and practice clean-code,
  • Become a master of Git,
  • Increase self awareness and work on your soft skills, teamwork shall improve.

I wish you success, fulfillment, happiness and fun.

Impediment List is of Tremendous Importance

Impediment List is of Tremendous Importance

2017 11 01 head

An impediment is any aspect hindering the team or a company to deliver value and higher quality to the customers. An agile organization continuously removes impediments, it is one of her major activities. This activity is ongoing and can never be completed.

Impediments resolution is Kaizen 改善, a key technique to lean, agile and Scrum approaches. The translation is change for better. Often agile organizations forget to aggressively identify and resolve impediments. We provide hints to improve your Kaizen process.

Identify Impediments

The first step is to identify and classify your impediments. There are always impediments; solving them will increase effectivity of your department. You shall

  • Make impediments visible. Publish them on the Scrum board for all to be seen. Use diagrams and colors. Do not hide them in an issue tracker or a spreadsheet,
  • Raise issues early and often. The daily stand-up meeting is an ideal platform. Ask each day if impediments jeopardize the sprint goal,
  • Feelings matter. Let everybody identify impediments. An impediment is a dysfunction of the organization, never a flaw in an individual,
  • Aim for effectivity and quality. Efficiency is less important,
  • Find out the root causes, discuss the 5 Whys over the how,
  • Provide safe to fail environment. Use the Plan-Do-Check-Adapt loop to implement your experiments to improve,
  • Life the Scrum values. Lead by example: Focus, Commitment, Transparency, Openness, Respect, and Courage.

Solve Impediments

The real work of the agile coach is eliminating impediments. The goal of resolving impediments is often higher internal quality - internal quality is the sole responsibility of the Scrum Team -. Work on improving external quality - a delighted customer will order more -, and improved effectivity.

Start with short-term wins to prove the organization that it is worth the effort.

Allocate time and resources in each sprint to resolve a few high prioritized impediments. Please define measurements Key Performance Indicators KPI. Find out if the solution solves the root cause of the problem.

All team members are responsible to solve team impediments. The Scrum master is responsible to coach the team and help solve overall impediments rooted in the organization.

Track Impediments

The identification step is the easy one, do not forget to track impediments and their resolution. You have a serious problem

  • If the impediment backlog lives in the mysterious black book of the Scrum master,
  • If your impediment backlog does not change,
  • If your impediment backlog is empty,
  • If you have an impediment backlog with a growing number of active impediments,
  • If the Scrum master resolves all impediments himself,
  • If you do not solve at least one impediment each sprint - it is now a mandatory part in the Scrum Guide 2018 version -.

As a Scrum coach, Scrum master or Scrum team member you should reflect upon the above statements at least monthly. Otherwise you are just pretending. Remember

Doing Agile instead of being Agile is just Cargo Cult.

Agile Approaches and Fix Price Contracts

Agile Approaches and Fix Price Contracts

2017 10 01 head

The difficulty with contracts is that it is about trust. Here lays the roots for success or disaster.

If no trust exists the henceforth dread process is established. After tough negotiations the development team starts but does not collaborate with the customer. They just build what is written in outdated requirements. A subpar solution is shipped and the relationship with the customer is deeply damaged. We can do better. We shall collaborate, trust each other and create an awesome product. This is the essence of being agile. How shall we be agile and work the lean way in a contractual environment? By contractual I mean no time and material approach.

We shall

  • Define and rapidly realize the minimum viable product MVP. It defines success and build trust. Use this MVP as anchor during contract negotiation.
  • Prioritize Money, Time, and Scope. The most important item - and it can only be one - is the basis for the agile contract,
  • Service is the essence of software application development. So your agile contract shall be a service contract. Select a rolling contract approach to deeply involve both sides,
  • Fail early and learn. The disputes between contractor and customer shall stay reasonable due to the smaller amounts in play.

Contract

Initial success is the timely realization and deployment of the MVP. The MVP is the anchor of trust. Use rolling contracts to extend the product.

Priority on Money
Frame cost as investment and discuss value with the customer, not just money.
Fix the budget - money dimension - and have some tentative milestones. Therefore we must invest to prioritize the stories and talk about your MVP. Often nice to have functions are never realized.
Priority on Time
Sometimes, it is not about the cost. Instead, there is a deadline, like the end of the fiscal year.
Fix the product milestones - time dimension -. We still must define a budget and prioritize the stories. The must stories shall be implemented before deadline and budget can be extended to respect the deadline.
Priority on Scope
And in other cases, the scope actually is quite defined, and both money and time are flexible. In these cases, it makes sense to fix the scope variable.
Fix the functionality - scope dimension -. Budget overrun and delays are possible. Defining the scope in advance is less agile. You assume you already know everything about the product to build.

The contract defines a service. The delivered product is the output of the service contract but not the sole item of the contractual binding.

Tracking

Signing a contract is the easy part. Now you must track progress, manage changes, and reach the agreed goals in a timely and economical way.

  • Commit in trust, training, and decision making. Train your people and customer’s representatives,
  • Deliver incrementally every few weeks to prove progress and cement trust,
  • The more trust you build, the less escalation you will have during the product,
  • Have rolling planning, budgeting, and tracking. You shall gain visibility of progress, transparent costs, and tentative end for fulfilling the contract,
  • How do you track and document changes?
  • How do you inform your stakeholders about contract changes?

The later questions are solved differently based on the level of trust between the contractor and the customer. Try to avoid a full fledged change management process and systematic escalations.

Work Approach

The Agile Manifesto states

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

Agile Manifesto, 2001

Here the approach to define and finalize a fixed price agile contract. Please remember the manifesto, it is all about collaboration and responding to change.

  • Create the product vision and initial backlog together. Write good epics and identify main topics - topics identify the minimum outcomes to maximize solution value -,
    • Commit to the vision and defined MVP,
    • Define a budget for the product,
    • Define tentative deadlines,
    • Refine some epics and write the stories. Keep them concise and clear,
  • Estimate the product backlog together with the customer,
    • It is all about cooperation and adaptability,
    • Use relative estimates to classify stories, use story points,
    • Talk about business value and implementation risks,
  • Finalize the contract, What happens if costs overruns happen -. We suggest to share the costs and define the percentage each party shall pay, if the supplier pays 0% it is a time and material contract. If the supplier pays 100% it is a fix price contract. Aim for 50% -,
    • Define a checkpoint to validate the estimates and hypotheses,
    • Define exit criteria and exit points for both parties,
    • State governance how to simplify scope and stories to respect budget. State and agree upon escalation process if no agreement is found,
  • Invite the customer to the Scrum sessions. Sell the entire Scrum team and not individuals,
  • Sell releases containing a small set of sprints,
    • Deliver and deploy the build solution,
    • Have the end users use the deployed product.

The Scrum master, the product owner and the team shall perform these activities. Never use external consultants or business analysts. The ones writing the stories and estimating them shall implement them.

Fallback

Hide the fact you are working the agile way. Don’t tell the customer you are working any differently to normal. Clearly state internally why you do it and why your corporate values allow this solution.

Estimate and plan the work as you would normally, sign a perfectly normal contract. UseAgile techniques and especially eXtreme Programming to improve delivery. You need to have a don’t ask, don’t tell type policy because basically you are lying.

Conclusion

The most successful projects I worked for had selected the money dimension seen as investment budgets. Goals correction were communicated early and the contract amended accordingly. We avoided complicated and expensive change request processes.

The build products were very successful. We respected the agreed budget and were timely. The dynamic was in the scope definition. We delivered early and often high quality increments so the end users could adjust their expectations and refine their needs.

Your Management Job in an Agile Environment

Your Management Job in an Agile Environment

2017 09 01 head

Introduction

You develop your software products using agile approaches. You are doing Scrum, Kanban, Scrumban, Lean Software Development, or LeSS.

Your collaborators start asking questions about organisation, management roles, and critique the internal processes. How employee yearly evaluation, bonus evaluation - work? You hear strange statements such Be agile, do not do agile.

You as a manager realise you have to describe your role in an agile environment. What does it mean? Please read first this blog post describing What is an Agile Company?

If you have time read the post Why Scrum Masters are not Administrators?

Another one is about So Calle Agile and Scrum Failures. Upon reading, you shall have a deeper understanding how to describe your role as a manager in an agile environment.

We will first discuss your work environment, and second characteristics of people you are working with. Finally we will sketch your job description.

Complex Systems and Empiricism

The development organization of a software solution is a complex system. This means that nobody understands the rules. You cannot implement a measure and be certain it will improve the whole system. If you want to implement lasting improvements you must

Provide transparency
The more you and your collaborators know the better your decisions are. All information shall be available to all participants, including financial and marketing data,
Experiment with improvement
Inspection and adaptation are keys; processes and value streams shall constantly be improved, Take the lean approach: Relentlessly eliminate waste and increase the value stream,
Establish long living teams
Teams are the backbone of your company; people are human beings not resources.

Collaborators and their Needs

Knowledge workers are the collaborators you hire in this millennium. They are smart, motivated and better qualified than you are.

It doesn’t make sense to hire smart people and then tell them what to do; we hire smart people so they can tell us what to do.

Steve Jobs

A manager working with knowledge workers has understood the above quotes and

  • Believe in the X vs Y theory. You provide safety to all your people and truly respect them. People only perform if you provide psychological safety. The test is Do you compliment every collaborator you are working with at least once a week,
  • Provide a purpose, give autonomy, and let them achieve mastery. The test is Continuous learning and improvement through trial and error,
  • Promote exclusively intrinsic motivators. The test is No bonus or monetary incentive system exits in the company,
  • Establish a work environment where collaborators work in flow. The test is No email or phone answering rules in the company; no mandatory overtime.
Simon Sinek

The agile and lean principle Plan-Do-Check-Adapt applies to agile managers as well.

  • Are you catalyzing organization change to embrace agile values, starting with a culture of value delivery?
  • Do you provide significant organizational roadblock removal for agile teams?
  • Do they perceive you as a coach and leader more than as a manager?
  • Do you use metrics to support teams improve their performance and to help senior leaders understand how to improve value delivery?
  • Are you helping your partners create lean processes to synchronize with agile teams?
  • How are suppliers encouraged to work in an agile way?

Let Go and Delegate

Delegate and empower your team. As a manager you shall reconsider which activities are your responsibility. An agile team must at least be self-organized and ideally self-designed.

manager lead
The team performs the work,
self-organised / self-managed
The team organises and performs its own activities and work,
self-designed / self-selected
The team creates, selects and removes new members and maintains itself. Logically they are also in charge of evaluation, rewards, and career development,
self-directed / self-governed
The team has no outside management.
2017 09 02 types of authority

See also Mike Cohn’s blog https://www.mountaingoatsoftware.com/blog/two-types-of-authority-leaders-must-give-to-self-organizing-teams [Types of Authority Leaders must give to self-organizing Teams].

Job Description

Champion the Agile Process
as a manager, you will be expected to get behind the move to agile and support it in both your words and your actions. Create an awesome work environment to attract smart collaborators,
Mentor Teams to Solve Problems
Their confidence, learning, and the level of respect they earn from their teams will depend to a large extent on how you treat them,
Participate in transition planning
Agile transformation promotes self-organized teams that work efficiently on their own, which involves strategic planning,
Find ways to reduce waste
as an agile manager, you will be expected to continually improve workflow and maximize efficiency by eliminating impediments. This key activitiy may not be part of your current role as a manager,

You will have more than enough work as a manager in an agile company. Only the content will be different.

Scrum Masters are not Administrators

Scrum Masters are not Administrators

2017 08 01 head

Lately I was asked if I was interested to work as Scrum master. The below job description was handed me over. What an experience it was.

The text with the regular font is the job description. I wrote down my comments in italics.

  • Manage team of software engineers: provide technical direction, schedule and delegate work, evaluate performance. Advise on best practices and developing practice, review code.
    • A Scrum master empowers a development team. The team schedules the work during each Sprint. The team members select the stories they will implement next. The Scrum master advices best practices, but he really should not perform code reviews and police the team.
    • He does not manage and certainly not micro-manage engineers. He shall never evaluate performance of team members. He shall not police the team by reviewing code.
  • Analyze requirements and provide time estimates: consume non-technical functional requirements documentation. Provide feedback when necessary, translating into the technical documentation when necessary. Provide feasibility assessment and estimate level of effort to implement
    • The product owner and the team writes and refines stories. Avoid requirements and move to stories and epics. Technical documentation and feasibility assessments shall be done by the team. Only the team has the right to estimate the effort needed to implement a story.
    • The Scrum master is neither a business analyst defining requirements, nor a product manager creating schedules and time estimates.
  • Help develop, communicate and enforce workflows. Work with Development department head and QA to assure that clear workflows are established and followed within the team
    • An agile team has quality assurance roles as cross-functional capabilities. You do need a quality assurance team. Dissolve the quality assurance department. Move their specialists to the teams and get rid of the vice president of quality assurance.
    • The Scrum master does neither administrate flow of information nor implement processes. He trusts and coaches his teams to deliver best quality.
  • Coordinate with QA, IT Operations, and Database Administrators on testing. l Launch planning, launch, and troubleshooting/optimization of features post-launch.
    • The product owner plans the release schedule with the team. A high quality product does not need a troubleshooting process. The above nicely describes an organization based on functional silos. No cross-functionality is ensured and product focus is an unknown concept.
    • The Scrum master is neither an administrator nor an overall coordinator. His teams are self-organizing and directly communicate with all involved parties.
  • Architecture: participate in technical design of new and expanding systems and infrastructure. Aid in creating documentation for new and existing systems
    • The team grows the architecture and implements the solution. Not an external so-called expert shall be in charge of the architecture or sole owner of the design.
    • The Scrum master is neither in charge of the architecture nor the external expert imposing his views on the team.

I clearly decided not to apply for this position. I was fascinated that companies still believe that such alibi positions will not be detected. The above position is a flag for me. As an agile developer, Scrum master or product owner I shall never work for this company. I will not have Fun at Work.

They were looking for an administrator and a classical project manager. They were not looking for an agile professional and Scrum master.

I kindly suggested the authors of the job description shall get an introduction training to Agile, Lean and Scrum. I will certainly not recommend this company to my acquaintances.

What does a Scrum Master?

What does a Scrum Master?

2017 07 01 head

What does a Scrum master the whole day? Are you just a fancy administrator or are you actively removing impediments? Do you coach your teams toward mastery?

Too often Scrum masters become Scrum administrators. They schedule meetings, moderate discussions, write progress reports and draw beautiful slides how to improve the organization. Or worst they collect extensive statistics about velocity, planned versus delivered story points or how precise story estimation was.

At the end of the day your job is to mentor your team to be champions and build awesome products. Wake up! You are not a product manager, you shall be a catalyst.

Facets of a Scrum Master

  • Be a servant leader whose focus is on the needs of the team members and those they serve (the customer), with the goal of achieving results in line with the organization’s values, principles, and business objectives,
  • Teach ensuring agile, lean, Scrum and other approaches are understood and enacted,
  • Mentor to transfer agile knowledge and experience to the team, and concentrate on continuous improvement,
  • Coach the individual with a focus on their mindset and behaviour,
  • Help the organization to truly collaborate with the Scrum team,
  • Be an impediment remover solving blocking issues to the team’s progress, taking into account the self-organizing capabilities of the development team,
  • Facilitate by setting the stage and providing clear boundaries in which the team can collaborate,
  • Be a change agent to enable a culture in which Scrum teams can flourish,

Remember the definition of an agile or Scrum team. Ask yourself if your team fulfill this definition. If not work hard to improve. Each team is self-managing, cross-functional and customer-focused, co-located, long-lived.

Scrum Master Tasks

  • Help your teams to remove impediments. From time to time you could also remove a specific impediment.
  • Evangelise agile, lean values and principles
  • Moderate Scrum meetings - do not forget to regulate delegate moderation to your team -
  • Gently transform your department and organization to become more agile
    • Remove yearly evaluation, practice bi-weekly face2face meetings
    • Remove any individual bonus, either introduce team bonuses or better remove extrinsic motivators
    • Remove test and quality department, move the specialists to the Scrum teams

You shall by all means avoid administrative waste such as * Writing information in Word documents or in Wiki that nobody actively uses, * Writing multiple times the same information, * Writing status information instead of publishing it on the team board. The board shall be the information radiator of your product development. Do not create head sinks in documents, * Holding meetings with one person projecting their notebook screen content and all other attendees looking at it -.

Tips and Tricks

A good Scrum Master helps a Scrum Team survive in an organizations’ culture. A great Scrum Master helps change the culture so Scrum Teams can thrive.

Geoff Watts, Scrum Mastery
Face 2 Face
Prefer physical meetings to all other communication medium. Use video conference and chatting tools to feel the mood of your team,
Emergent
Good enough to try it, Safe enough to do it. Do not look for the perfect solution before trying a new approach,
Plan Do Check Act PDCA
Embrace complex systems and try new approaches to improve your team. F focus on external and internal quality,
Process Ownership to Keep It Alive
You are not in charge of the team processes, the team is,
Community of Practice PoC
is an effective way to move your team to technical excellence.

If you are new to the role read "The Great Scrum Master" from Zuzana Šochová. A more extensive list of books can be found here.

So Called Agile and Scrum Failures

So Called Agile and Scrum Failures

2017 06 01 head

Agile will never guarantee product success. All projects, especially application development, entail risk. If a product was risk free it is unlikely to provide significant benefits or competitive advantages. There are however, a number of ways in which Agile Scrum projects repeatedly fail which are worth examining.

Agile approaches and Scrum framework can reduce risks in projects and increase the return on investment. Standish Group report is a proof for this statement. One picture shows it all - look at the failed column -

2017 06 01 agile vs waterfall

Below are common patterns and errors when doing agile instead of being agile

Wagile
is a team which continues to follow basically a Waterfall product but uses the language of Agile and adds a few agile practices. For example, the team may present burn-down charts together with Gantt charts. Or the team compute individual resource allocation for the next three months. You can also replace Waterfall with V-Model, HERMES, RUP, and even sometimes SAFe.
ScrumBut
describes a team which claims to follow Scrum but misses various practices; for example We do Scrum but we don’t have a Product Owner. Or We do Scrum but the Project Manager allocates tasks. Such teams normally have a long list of “buts” and show little progress of removing them.
Hitting The Scrum Wall
The most popular Agile method is Scrum which is a product management technique. Scrum is normally used with a number of other Agile techniques, typically user stories redaction and the technical practices from eXtreme Programming – such as TDD, ATDD, code refactoring, continuous integration and delivery CI/CD, pair programming, etc. Teams that adopt the Scrum framework initially see an improvement in productivity and customer satisfaction. However without the technical practices quality is low, and the team hit the wall. The quality gap makes it impossible to maintain the pace in the long run.
Fake Agile
occurs when a team declares itself Agile and blames everyone else for their failure to interact correctly with the group. Such a group typically stops writing documentation, listening to business analysts, product managers and other customers, and dictates its own delivery schedule. Meanwhile the team do not improve quality, does not adopt test driven development or any other practice they dislike.
Potemkin Agile
occurs when an a team adopts and applies an Agile method well but does not deliver business value. This is a form of goal deferment were the team consider adhering to the process rather than delivering business value as the success criteria.
Customer (Business Analyst, Product Manager, Product Owner) overload
on a well functioning Agile product the customer, or proxy customer, is called upon to do a lot. They need to decide requirements, set priorities, scout ahead of the product, align strategy, work with the developers, testers and managers, and may even have their own day job to do. In the earliest XP product (“C3”) the first business analyst came close to a nervous breakdown.
Fall back
management may bring in consultants and other experts help switch a team to Agile. When the consultants leave some teams return to their old ways of working. Advisers and consultants can be a great help when introducing Agile but they need to build capacity in the development team to continue learning and evolving when the consultants are gone.
Failure to go far enough
To maximise the benefits of Agile Software Development the people, processes and organization that interface and work with the Agile team need to understand Agile and adjust their expectations and working techniques too. Agile is not a drop-in technology that can be swapped in to replace another failing method. Isolated Agile teams will find it difficult to be completely Agile. When other groups adapt the benefits of Agile can spread beyond Software Development.
Exploding cards
happens when teams do not sufficiently understand the technology they are working with – either in the solution or problem domain. Small work packages suddenly turn out to be large tasks in their own right.
Hyper changing requirements
Most Agile methods, especially Scrum, hold the iteration goals fixed for a few weeks. An exception is Kanban. Most businesses should be able to hold to goals for such short periods of time. If it proves impossible to hold requirements and goals fixed for even one week then something is wrong. In a few cases the business is genuinely changing extremely rapidly. In this case teams are better off using Kanban style management than a Scrum based approach. More often hyper change in goals and requirements are a sign that something is wrong beyond the team. The organization itself may lack strategy and objectives, or the Product Owner may not be filling their role adequately.
Fragile, not Agile
some of the Agile techniques, like TDD, ATDD, CI or CD, when poorly applied with a lack of understanding can show short term benefits but create long term problems.

Few of these failure modes are unique to Agile approaches, they are reoccurring failure modes for all IT software development projects. Neither are they a comprehensive list of the ways in which Agile or traditional application development projects fail.

We can state that (see the Scrum Guide)

  • Team process improvement is a major focus for the Scrum Master or the Scrum Coach
    • Coaching the Development Team in self-organization and cross-functionality,
    • Teaching and leading the Development Team to create high value products,
    • Removing impediments to the Development Team’s progress,
    • Facilitating Scrum events as requested or needed,
    • Coaching the Development Team in organizational environments in which Scrum is not yet fully adopted and understood.
  • Company process improvement is a major task for the Scrum Master or the Scrum Coach
    • Leading and coaching the organization in Agile and Scrum adoption
    • Planning Scrum implementations within the organization;
    • Helping employees and stakeholders understand and enact Scrum and empirical product development;
    • Causing change that increases the productivity of the Scrum Team; and,
    • Working with other Scrum Masters to increase the effectiveness of the application of Scrum in the organization.

The essence of succeeding with Agile, Lean and Scrum is

  • It is a change process with well known and discussed aspects,
  • You must have a strong and experienced Scrum Master and Scrum Coach to maximise success,
  • Do not tinker with the Scrum process before you really master it,
  • If you have to scale your process, please consider LeSS.

Git Local Repositories for the Impatient

Git Local Repositories for the Impatient

2017 05 02 head

Start using Git

You shall put your source code under a version management system. The actual industrial standard is git, a distributed version control system - DVCS -.

Install git on your development machine. In case you are using IntelliJ IDEA configure git in the IDE through the preferences' pane.

To put your product under git version management, go to the root of the product and perform the command

git init

To add a specific file or all your source files to git

git add [filename]
git add *

You can perform these operations to add a product to a local Git repository directly in IntelliJ IDEA as follow

  1. Open the product you want to store in a repository.
  2. On the main menu, choose VCS | Import into Version Control | Create Git Repository.
  3. In the dialog that opens, specify the directory where you want to create a new Git repository.
  4. Put the required files under Git version control. The files appear in the Local Changes tab of the Version Control tool window, under the Default change list.

Commit Changes

To commit your changes you simply

git commit -m "commit message, should be clear and legible"

You can perform these operation directly in IntelliJ IDEA (VCS functions) as follow . Switch to the Version Control tool window and switch to the Local Changes tab. . Expand the Un-versioned Files node, and select the files to be added. From the context menu, chooseAdd to VCS, or press ⌥⌘A. . Switch to the Project tool window and select the files to be added. From the context menu, choose Git | Add or press⌥⌘A.

Each time you commit your changes you gain the ability to reverse to exactly this state. Each time you made a modification and tested it, just commit it. The cost of a commit is marginal. When working with Git, TDD, and ATDD it is normal to commit every few minutes. By few minutes we mean 5 or 10 minutes.

Observe yourself. If you commit at the end of the day, you are using your DVCS as a backup medium. It is time to change your habits. Use Git as an history of all successful changes you implement, and simply rollback all unsuccessful ones, simply discarding them.

What is Your Project Status?

To find out what the status of your product is, simply

git status

The same information is available in IntelliJ IDEA under

  1. Open the required product
  2. On the main menu, choose VCS | Refresh File Status
  3. Switch to the Version Control window and open the Local Changes tab.

Always Work with Trunk

Ideally you shall always work against trunk also called main branch. Because you develop using TDD and ATTD approaches you know your source code is always working. This approach is deeply compatible with lean and agile values. It is also the one with the least waste of effort.

If your team decides to work with branches, make them short lived!. See How to Git Branches for the Impatient post.

Discarding Changes

You find you the changes you made locally were not a good decision. No problem, with

git reset --hard

You revert to the last save committed set of files. Resetting with the hard option recursively discards all of your currently uncommitted (unstaged or staged) changes.

You want to restore just one file to its previous committed state.

git checkout --[filename]

Configuration Tips

You shall avoid end of line character warnings by configuring git to handle them. The situation arises because Microsoft OS uses CRLF for end of lines instead of CR.

You can configure git to handle it by running on Windows.

git config --global core.autocrlf true

or on Linux and macOS

git config --global core.autocrlf input

When using an IDE consider the Editor Configuration approach for end of line, identation and tabs versus spaces.

What is Our Specialty as Agile Coaches at tangly?

What is Our Specialty as Agile Coaches at tangly?

2017 05 01 head

You have currently more agile coaches on the market than sand corns on a regular beach. They all promise the holly grail of agile product development, highly productive teams and perfect applications for internal and external customers.

Why should any sane customer hire us to deploy or perfect their agile approach?

Your overall goal is to continuously improve the products delivered to customers. I am convinced to achieve ingrained success an agile coach shall

  • Coach team members how to reach technical excellence. They build your solution, it shall fulfill the user’s requirements, be of high quality and affordable.
  • Learn them to attain mastery and build awesome products,
  • Coach the team how to improve, focus and enjoy the work on a daily basis. Flow shall be natural for them,
  • Coach the organisation how to adapt, provide the ground to have the best collaborators and become most successful in the market. Use a lean approach.

In other words you need technical coaching, team coaching and organization coaching. Avoid hiring management consultants with an agile painting, coaches should understand software and product development.

At the end of the day your products shall improve. Your customers shall be delighted. How your highly qualified teams develop software shall become more effective and efficient.

First comes quality, effectiveness and only later efficiency. Look at lean methods, first we improve the quality - effectiveness for the customer - than the productivity - efficiency for the company -.

Minimum Viable Change

What is coaching?

Coaching means that an experienced coach will work closely together with an organization to improve the organization’s product development.

This usually involved shortening feedback loops and focusing on improved quality.

This faster feedback cycle leads to benefits such as higher productivity and more flexibility.

https://less.works/[LeSS]

How shall you coach?

You use Minimum Viable Change (MVC). So I am using yet another acronym. A MVC has a hypothesis how to improve an aspect of your organization or your team. It has measurements - often called KPI by management - to decide if the change was a worthy improvement. For the savvy readers it is a variant of PDCA, a cornerstone of all agile and lean approaches.

Our core strength at tangly is we truly understand software technology and software craftsmen. We consistently connect organizational MVC with technical MVC. This is our special recipe for deep-rooted and lasting success.

Three Ways of Coaching

To improve your organization you need these three levels of coaching

Organizational coaching
The coach works with multiple teams and the management to improve the organization and its structure
  • Change processes to be customer centric,
  • Improve organization to see the whole product and the customer,
  • Delegate decision-making processes to teams,
  • Remove functional silos,
  • Redefine the roles of product managers, middle managers, quality managers,
  • Eliminate waste: processes and artefacts not adding any value to the customer products.
Team coaching
The coach works with one or a few teams to improve their team-working and LeSS practices. It is common for a coach to take on the Scrum Master role.
  • Implement MVC,
  • Improve Scrum approach, teach agile values,
  • Introduce eXtreme Programming practices,
  • Improve decision making, meetings, flow of customer value,
  • Learn, teach, coach through the stages of Shu Ha Ri (守破離),
  • Empower the team to turn into best of breed.
Technical coaching
The coach works with (or on) a team on their actual codebase in order to improve the technical practices and adopt agile development techniques such as simple design, refactoring, unit testing, test-driven development and acceptance test-driven development.
  • Discovering “code/design smells”. Places where code/design could be improved,
  • Explaining modern, “clean” code that is simple and easier to change and maintain,
  • Refactoring “smells” into clean code,
  • Test-driven development and test automation,
  • Continuous integration and continuous delivery,
  • Specification by Example (Acceptance Test-Driven Development),
  • Efficient and effective working practices (IDE, automation),
  • Applying design patterns.

Thoughts

I believe that the right mindset is boosted by good practice, just as good practice cannot be achieved without the correct mindset. When advising others, I spend much of my time trying to connect practice with mindset, as they are symbiotic. You truly need both.

Choose wisely your coaches. Check they can coach at technical, team and organization level. Senior coaches shall have a decade of experience. Good coaches practice Gemba and should restrain drawing too many slides.

Agile Trends Switzerland 2017: We Won

2017 04 02 head

It is time to throw a big party in Switzerland. More than 60% of all projects are now realised using an agile approach (see Agile Trends Switzerland 2017 by SwissQ), Scrum being the most popular one, Kanban as the second one. What a change in just one year. n 2016 we had 40% of projects being agile, one year later we have 60%. Time to adapt and become agile.

It was a long journey. At the beginning of this millennium I was convincing managers that agile is not evil and could be used in Switzerland. And Yes, agile is compatible with Swiss culture - we finally added Scrum to the national methodology Hermes -. The report 2016 patently states that agile projects are more successful for the customer and the company than non-agile ones. It is genuinely time to adapt.

See our previous blogs for the results of Agile Trends Switzerland 2012, Agile Trends Switzerland 2012, and Agile Trends Switzerland 2013. They illustrate the slow gains of agile and lean approaches in our country.

But I have a strong déjà-vu. I am now convincing managers that agile approaches for the whole company are not evil and could be used in Europe and Switzerland. Yes, agile approaches are compatible with European and Swiss cultures. Yes, holistic agile approaches increase the odds of success to deliver tremendous products to our customers. Again I hear the same arguments as 15 years ago why our companies are different and need special approaches.

To ripe the benefits of agile approaches it is now time to transition whole departments and the company. I hope to succeed in the first half of this century.

Below some findings of the study.

Major Hurdles Introducing Agile Approaches

The major hurdles when introducing agile approaches at department/company level are

Current company culture
First change the structure, it enables the cultural change. Quite a few companies try to change their culture without altering their structure. It is doomed to fail,
Existing company structure and frozen hierarchies
It is time to rethink your structure and move decision power to the teams, near to the customers,
Controlling not tailored to agile approaches
It is difficult to be slightly agile. The same can be said about lean. Either you are lean, agile or you are not. Try Beyond Budgeting Round Table,
Overall processes not tailored to agile
Once you change your structure and hierarchy, the processes will evolve,
Trust is missing
Here again we talk again about trust or the believe your collaborators are competent and do their best to achieve success. As an organization and as a manager it is time to decide. Do you truly trust your collaborators?
Customers are not involved or engaged
Customers are the center in the new world. Please involve them, talk to them, find out what they need

You could also say organizations are still struggling with Being agile, not doing agile. Avoid cargo cult.

Our Recommendations

You shall start introducing agile approaches at department and company level.

Senior management is fully committed to the approach and approves the cultural change. Read the book "Reinventing Organizations" from Frederic Laloux to understand the implied changes.

Train your managers to understand agile, lean, and customer centric approaches. And training is not half-day introduction, it is days and days of training, workshops, and reading.

The transition is painful, see the seminal articles of Steve Denning Why do managers hate agile? and More on why managers hate agile.

The principles are

  • Products are value driven and quality oriented
  • Teams are focused to master their work and tools - Craftsmanship, eXtreme Programming -
  • For huge development departments consider using the LeSS approach.
  • Teams work best if you can answer a sounding yes to these five questions.
    • Psychological safety: Can we take risks on this team without feeling insecure or embarrassed?
    • Dependability: Can we count on each other to do high quality work on time?
    • Structure & clarity: Are goals, roles, and execution plans on our team clear?
    • Meaning of work: Are we working on something that is personally important for each of us
    • Impact of work: Do we fundamentally believe that the work we’re doing matters?

If you have doubts please refrain using agile and lean methods. Introducing agile approaches in product development is similar to introduce lean principles in a production environment. Either you do it fully or the benefits are marginal. Worst the best employees will see through and probably leave due to unkept promises.

Fun at Workplace

Fun at Workplace

2017 04 01 head

It is Spring in Switzerland. We often have time for small talk because it is cold or rainy outside. Quite a few of the discussions are about job opportunities and how interesting and motivating the current activities are. I selected two articles to help everybody to decide if your software development job is worth the effort.

Just go through the questions and ask yourself if the current product provides the current settings.

Joel Test

The Joel Test is slightly outdated. Keep his questions and just do the following changes

  • Can you replace CVS with Git to reflect the current state of the industry?
  • Move from daily builds to continuous build at checkin.
  • Remember "Daily Builds are for Wimps"
  • Do not only fix bugs but also write the associated automated unit and acceptance tests. Remember Defect Driven Development to guaranty your customer he will never again see the same issue
  • Check that your specification also has acceptance criteria which can be automated
  • For Microsoft developers, are you forced to use TFS instead of tools of your choice and why?
  • Are you testers and quality experts integrated in your Scrum team and sitting in the same room? Do you think about tester and quality as roles and not persons?
  • Do new candidates code and perform refactoring to achieve clean code during their interview?

8 Questions to identify a lame programming job. The questions are up to date. His final remark is worth remembering

No matter how great the potential projects and teammates might be, I don’t think you can do truly meaningful work in an environment where you, the developer, aren’t empowered to succeed.

If a company doesn’t get that, then they don’t get the software.

My personal list

My personal list is:

  • Is respect a key value of your organization and your team?
  • Are you creating a meaningful product and improve quality of life of mankind?
  • Are you working in an agile team, preferably a Scrum team?
  • Has your team as sustainable development rhythm?
  • Are you using good software practices daily such as TDD, ATDD, refactoring, check-in in Git master, CI/CD, deliver each day?
  • Can you contribute to the open source projects you are using during work time?
  • If you are developing in Java are you using IntelliJ IDEA as IDE?

Take the ten minutes to decide if your current job is good enough. If not, it is time to regain ownership of your life.

List of Agile and Lean Books for Software Engineers or Students

List of Agile and Lean Books for Software Engineers or Students

2017 03 01 head

Purpose

I am regularly asked by practitioners and technical universities which books I shall recommend to learn more about agile developers, agile projects, and agile companies.

Below a selection of books I read and found really interesting. The selection is certainly not exhaustive. The criteria are

  • I read the book and learnt something
  • I have bought the book to have it as reference, I mean a real reference opened at least monthly
  • The book is available as ebook - I only read ebooks and save trees -

Software Engineering

  • Scrum and XP From the Trenches: How We Do Scrum, Henrik Kniberg A short and very good introduction to Scrum and XP, worth to look again and again, Kniberg is also the guy behind the Spotify Way
  • Clean Code: A Handbook of Agile Software Craftsmanship, Robert C. Martin The reference book about clean code and XP practices. You are a programmer, you know this book
  • The Clean Coder: A Code of Conduct for Professional Programmers, Robert C. Martin Are you a coder doing clean code (or are a looser)?
  • Refactoring: Improving the Design of Existing Code, Martin Fowler You cannot be agile and not doing refactoring, see also XP practices, You shall refactor daily or you are lying to yourself
  • Agile Project Management with Scrum, Ken Schwaber A seminal work from one of the founders of Scrum
  • Agile Product Management With Scrum: Creating Products That Customers Love, Roman Pichler From the expert for product owner role and activities, Make the shift from project to product
  • Agile Testing: A Practical Guide for Testers and Agile Teams, Lisa Crispin and Janet Gregory What does it mean to test and insure quality in an agile environment?

Agile Management Aspects

  • Agile Management: Leadership in an agile environment, Angel Medinilla An overview about agile management at company level, you should also read the "Leader’s Dilemma" and see the BBRT Beyond Budgeting Round Table ideas.
  • Leading Lean Software Development: Results Are Not the Point, Mary Poppendieck & Tom Poppendieck Lean means often to change your point of view to better understand the system
  • The Lean Mindset: Ask the Right Questions, Mary Poppendieck & Tom Poppendieck What does lean means?
  • Fearless Change: Patterns for Introducing New Ideas, Mary Lynn Manns Ph.d. & Linda Rising Ph.d. When introducing changes you have to understand change management. You call later read the works of John P. Kotter
  • Agile Retrospective: Making Good Teams Great, Esther Derby & Diana Larsen & Ken Schwaber The major place where improvements are identified is during retrospectives, often the less well-managed meeting in a Scrum team
  • Practices for Scaling Lean and Agile Development, Craig Larman & Bas Vodde A set of practices and ideas how to do big agile projects, the guys behind LeSS
  • Scaling Lean & Agile Development: Thinking and Organizational Tools for Large-Scale Scrum, Craig Larman and Bas Vodde A second set of practices and ideas how to do big agile projects, still the guys behind LeSS
  • The Leader’s Dilemma: How to Build an Empowered and Adaptive Organization Without Losing Control, Jeremy Hope & Peter Bunce &Franz Rösli & Franz Röösli The modern reference text for the BBRT initiative

Presentation Techniques

You have to make beautiful presentation to make a difference!

  • Presentation Zen: Simple Ideas on Presentation Design and Delivery, Garr Reynolds Presentation Zen Design: Simple Design Principles and Techniques to Enhance Your Presentations
  • Blah, Blah, Blah: What to Do When Words Don’t Work?, Dan Roam

History

You have to know history or you will repeat the errors of the past generations.

  • All books of Tom deMarco e.g. Peopleware, The deadline, Slack, Waltzing with Bears
  • All books of Gerry Weinberg e.g. Becoming a Technical Leader, Are your Lights on?, The Psychology of Computer Programming
  • The Mythical Man-Month: Essays on Software Engineering, Anniversary Edition, Frederick P. Brooks Jr.
  • Good To Great, Jim Collins

Free Time

  • Vorkosigan Saga, Lois McMaster Bujold
  • Empire of Bones Saga, Terry Mixon
  • The Ender Quintet, Orson Scott Card
  • The Art of War, Sun Tzu

Enjoy the reading. Please enjoy life and your family.

Please leave a comment to improve this list.

Agile Trends Switzerland 2016

2017 02 01 head

What are the main hurdles to introduce agile approaches in Swiss companies. The last five years learn us that

  • Introducing agile company-wide is a cultural change process. Such a change takes time and sometimes hurts,
  • Without commitment of senior management, the initiative will fail,
  • You must teach, coach, support your collaborators through the transition. You need to believe in them and provide an environment where they can safely learn and experiment,
  • At the end what matters is collaborator purpose, customer satisfaction, business value.

SwissQ has published a "SwissQ Software Development Switzerland 2016". The study can be downloaded as PDF from their website. See a previous blog for the results of Agile Trends Switzerland 2013 and another blog for the results of Agile Trends Switzerland 2012.

Below some of the findings of the study.

Major Hurdles When Introducing Agile Approaches

The number left is the value for 2016, the number in parenthesis is the value found in the study for year 2013. Bold items are new in the 2016 study and have no 2013 values.

  • 63% (72%): Culture change
  • 61% (21%): Management support
  • 42% (28%): The team
  • 30% (-%): Integration of the business
  • 28% (25%): Fokus on business value
  • 19% (-%): Discipline
  • 17% (28%): Know-how collaborators

The trend is a clear move toward agile approaches. Companies seem less reluctant to introduce agile approaches, but often compromise using for example bimodal IT. A typical Swiss example is the government Hermes method mixing Scrum with Waterfall.

Major opponents such as senior managers or internal and external customers more and more acknowledge the success of agile approaches.

The majority of teams use Scrum or Kanban. The associated events and tools are used. The deeper values such as test driven development, acceptance test driven development, clean code or refactoring are often missing. The key is

To be agile, Not to do agile.

We wrote in previous blogs that introducing Scrum in the development department or in the whole company is a change process. The coaches should be trained in change management and have experiences with resistance to changes. Our experience shows senior management are often not ready to move along the cultural change path.

Recommendations

You shall start introducing agile approaches

  1. Senior management fully committed to the approach and approves the cultural change. Read the book "Reinventing Organizations" from Frederic Laloux to understand the implied changes.
  2. Products are value driven and quality oriented
  3. Teams are focused to master their work and tools - Craftsmanship, eXtreme Programming -

For huge development departments consider using the LeSS approach.

Teams work best if you can answer a sounding yes to these five questions (see google:rework).

Psychological safety
Can we take risks on this team without feeling insecure or embarrassed?
Dependability
Can we count on each other to do high quality work on time?
Structure & clarity
Are goals, roles, and execution plans on our team clear?
Meaning of work
Are we working on something that is personally important for each of us?
Impact of work
Do we fundamentally believe that the work we’re doing matters?

If you have doubts please refrain using of agile methods. Introducing agile approaches in product development is similar to introduce lean principles in a production environment. Either you do it fully or the benefits are marginal.

Seven Pitfalls with Agile or Scrum Methods

Seven Pitfalls with Agile or Scrum Methods

2017 01 01 head

The post describes a presentation I gave at a company internal technical day. It reflects situations we have seen in a lot of agile projects over the last years.

I assume that the Scrum approach is well introduced in your company.

You are already proficient with Scrum, eXtreme Programming, Clean Code, Code Refactoring, how to write stories and story maps, and techniques such as TDD, ATDD.

You are using Scrum well and can laugh about all these posts about Scrum-But(t).

Still misunderstandings about Scrum abound.

We will present common pitfalls seen in teams already applying Scrum; meaning teams using Scrum as an empirical process, holding the meetings as described in the Scrum guide, and producing all expected artefacts.

We want to increase your awareness and reflect how you can become better Scrum experts. We exhort you to eliminate these misunderstandings in your projects.

What is Scrum?

2017 01 01 Scrum The Scrum approach is clear, well documented and shall be done by the book.

The Scrum framework is straight forward. The possibilities inside are unlimited.

It is like chess, the game has a simple set of rules, the variants how to play a game are limitless.

This article is not about Scrum But impediments describing common errors how meetings are held, artefacts created or roles lived.

We are talking about misunderstandings once you have reached this first level of mastership. Before talking about misunderstandings we should remember the most important facet of Scrum. Scrum is all about ROI. Scrum was defined because the founders were convinced you can develop more effectively better products with higher value.

The essence of Scrum is ROI

2017 01 01 Essence All decisions in Scrum should be based on Return of Investment ROI.

Each time you doubt how you should do an activity, ask your team what is the ROI of your proposition?

Stakeholders want ROI. Each time you request budget from your stakeholders you should always remember.

you want stakeholders' money, convince them. Show your different solutions to the problem.

In other words we want Bang for the Buck

You shall fill all time-boxed meetings

2017 01 01 Meeting The agile manifesto states

Individuals and interactions over processes and tools

Customer collaboration over contract negotiation.

Agile Manifesto

Perhaps too often we interpret these sentences as * Respect people, have nice interactions and avoid any hard discussions, * Collaborate with the customer, never disagree and avoid harsh truths. Swiss people are well-educated. They always empty their glasses in the restaurant and have trouble leaving some wine in the glass. They also do not like conflict.

We often forget the Pareto rule, 80% of all solutions are found in 20% of the time. Is it worth the time to find a slightly better solution for the remaining 20% of the problems? In Scrum terminology "it is also the 20% less important" Tips "ROI: Meeting costs versus solved issues" Meetings cost money. A meeting with 8 persons and of a duration of 30 minutes costs in Switzerland around 600 Swiss Francs or 500 Euro.

Compute your ROI.

You shall have a cross functional team

2017 01 01 A Team Scrum teams try to be fully cross-functional and invest a lot of effort to reach this goal. They probably do it because it is written in all Scrum tutorials. Every person should be able to take a task from the Scrum board and implement it. It is like a soccer team where each team member can play all roles.

Are you doing peer activities in your company?

As a rule of thumb a good T-shape person Is master in one technical area, Has a delegate, a challenger and an apprentice, Care about the domain of his users.

You shall allow changes anytime

2017 01 01 Change Ahead Scrum is about agility. Therefore you have the right to change anything at any time, isn’t it? Your stakeholders need the changes now. They cannot wait until the end of the Sprint, a mere ten working days or two weeks of elapsed time. But Scrum also states we have a vision, features, a minimum viable product and a potentially shippable product. How often can you change these key concepts? What is the balance between agility and chaos?

Here again we are back to ROI computations.

As a rule of thumb to test your decision Uncle Bob stated in the "Clean Coder" book if you deliver an application with errors the only professional approach is to sign personally a check to the customer for the lost of income. In other words are you ready to change the user interface two hours before the sprint demonstration will be held?

You shall not perform up-front design Architecture emerge during the coding of the solution.

2017 01 01 Indian Village So teams state that - No architecture is needed before starting coding, - No enterprise architecture should be defined or look at, - No non-functional considerations are needed. Look at the picture. Could you design a village without knowing about the ground, the kind of population, do you need school, do they have flood in the area? They believe that refactoring will solve all problems. Architects are no more needed, we are all talented hackers.

As a rule of thumb Be honest: our systems are complex but they are no ground breaking work. Similar solutions already exist. I expect a talented team to provide an architecture with some prototyping in less than a sprint.

You shall write user stories during coffee breaks

2017 01 01 Meeting Writing user stories is easy and anyway nobody has time for - The product owner has better to do. He writes the stories during a coffee break or just before the start of the planning meeting, - Anyway just read the requirements, it is all written down, - The developers want to code, they have no time to write some user stories or improve them.

Scrum states the product backlog is the most important document in a Scrum product.

If not why? As a rule of thumb Writing quality user stories is as tough as writing requirements.

It is the same job!

Be honest: Developers cannot write clean requirements or design a clean user interface

You shall not train engineering practices

2017 01 01 Rope You shall not train engineering practices

  • The process solves all problems,
  • I want to code, I do not have time to become a craftsman,
  • Scrum is snake oil. It cures all illnesses and makes you immortal, For the older ones, do you remember CASE, CMM and ISO-9000.
  • The PROCESS promises that you will deliver high quality software on time, on budget with unqualified and cheap collaborators.

Do you really believe in snake oil?

Do you think that a collaborator can win a competition just be respecting a process. He must train every week to achieve and maintain a given level of skills.

You shall worship Scrum as the PROCESS

2017 01 01 Process Scrum is a framework. You can use it to manage different things, including complex product development.

Scrum is defined in the Scrum Guide and consists of roles, events and meetings, artefacts, and a set of rules binding them together. It is based on empirical process control and bottom-up thinking.

Call for Action

Eliminate these misunderstandings in your projects

Act using ROI

What is the risk?

The truth is complex, more blurred. The answer for your product cannot be stated in one standard rule set. We are talking about agile quality assurance, lean approaches and best practices.

A best practice should only be selected through its ROI.

Please look at the Software Craftsmanship Manifesto.

Not only working software, but also well-crafted software,

Not only responding to change, but also steadily adding value,

Not only individuals and interactions, but also a community of professionals,

Not only customer collaboration, but also productive partnerships,

That is, in pursuit of the items of the left, we have found the items of the right to be indispensable.

2016

What you do NOT need to do in Scrum!

What you do NOT need to do in Scrum!

2016 12 01 head

I often hear discussions asking who is performing the traditional product leader tasks in Scrum. This is in general an excuse to state that Scrum alone cannot work and classical product leaders are still needed.

Below a list of tasks you do NOT need anymore to do in Scrum.

  • Make commitments on behalf of the team about how much they can get done by a certain date or estimate effort for the team,
  • Coerce the team that the commitments made on their behalf are attainable,
  • Coerce the team to change their estimates,
  • Give direction to the team on how to implement the work,
  • Monitor the team’s progress to make sure they stay on schedule,
  • Step in and determine the solution,
  • Conduct weekly status update and face to face meetings with the team to surface issues and provide direction,
  • Provide motivation and push the team to work harder than they might want to use carrots and sticks,
  • Decide task assignments among the team members and follow up on tasks to verify completion. Force them to work during weekends,
  • Be responsible for the team doing the right thing at the right time in the right way,
  • Update Gantt charts for the iteration and release planning,
  • Finding out that requirements mean to explain team members what they should implement,
  • Review the documents and code to guaranty quality,
  • Explain upper management why the product is late and why you need additional resources,
  • Explain upper management why you cannot transfer a team member to another product,
  • Attend weekly progress and escalation meetings to explain in details the status of your product,
  • Define the priority of the next activities.

Now you as Scrum master or agile product manager have to perform important tasks

  • Help to remove impediments to increase velocity of team
  • Coach team to implement Scrum and reflect on their activities.
  • Plan-Do-Check-Adapt PDCA principle is built into Scrum with retrospectives and sprint reviews.
  • Coach product owner to deliver a groomed and refined backlog
  • Inform and support all product’s stakeholders
  • If you reflect on these two lists it becomes obvious that the first list describes malfunctions and the second one describes mechanisms how to improve continuously - lean principles -.

Jeff Sutherland gives some interesting insights about the Scrum master

  • The Scrum master enforces Scrum practices Coaching rather than command and control
  • The Scrum master effort is
    • Few problems: for a small team 10%, for a large team 50% of your working time,
    • Many problems: for a small team 50%, for a large team 100% of your working time until the team reaches a flow status,
    • A Scrum master is a change agent and supports the organization to become more agile.

I experimented with the Scrum master being part of the team, or Scrum master working for multiple teams. Experience has shown us that the full-time Scrum master is more effective.

And the LeSS framework has a clear rule about product managers

Keep product managers away from the teams!

Remarks

For all Project Management Institute +PMI_ fans, the institute has acquired Disciplined Agile Delivery DAD in 2019. The PMI-ACP is working on integrating agile approaches in the official PMI approach.

Found a Limited Liability Company in Switzerland

Found a Limited Liability Company in Switzerland

2016 11 02 head

I left the company I founded 20 years ago and needed a new home to work and have access to social insurances. I had to found a limited liability company with a capital of CHF 20'000.

Here the steps for the impatient.

Foundation

  1. Swissregistration wrote the statutes for the company and performed the foundation with certifying notary for CHF 780. Six hours after the first phone call they sent me the electronic documents. The next day the documents were delivered in paper form. It was really impressive.
  2. Before you can found a company you must deposit the capital to a special account. I choose Postfinance. It is a well-known Swiss institute and their fee is only CHF 145. They insisted I meet one of their collaborators to fill their forms – probably never heard about web forms -. I had bad luck and they needed three weeks to create the account and send payment confirmation. They were aware of the problem and very helpful but it still took three weeks.
  3. Upon reception of the payment confirmation, I sent all signed documents to Swissregistration. The next day the company was founded and I received the official paper documents a few hours later.
  4. I went to Handelsregisteramt Zug to register the company. They were very nice but told me they have a lot of work. Two weeks later they finally registered the company and sent me the papers and an invoice of CHF 780.
  5. A few phone calls later Postfinance converted the account into a regular account. They give me access to the founding capital minus their fee

During the waiting time I started to organise for the mandatory insurances you need in Switzerland

  • BVG – the best offer was from Axa Winthertur. They also help me because the company CRIF AG wrongly classified the activities of the new company. – They never contacted me and their NOGA classification has a huge impact on my costs. They are morons -
  • BU/NBU – the best offer was from Basler Versicherung When starting a software startup I suggest not concluding any other insurances

The government controlled social insurances contracts can only be initiated once the company is registered in the Swiss company registry. The AHV center needed more than two weeks to confirm my registration.

Lessons Learnt

  • The costs of founding my limited company in Switzerland is CHF 1'705 or 8.5% of the capital of the company.
  • The state owned companies and state services are very slow; in my case 20 times slower than the private ones. On the bright side their collaborators are very supportive.
  • Public or big private companies such as insurance firms are incapable to send documents or invoices electronically. They only use paper mail. Upon inquiry their collaborators told me their processes are incapable of handling modern communication ways.
  • Upon foundation you get pseudo-official letters from various companies to sell dubious services. Just throw them away.
  • The industry development office of Kanton Zug has no clues about start-up companies; the ones in Zürich and Luzern seem better.

Now we are ready to create great products with our customers

Please leave a comment if you know a better or less expensive way to found a Swiss liability company.

(the post was also published on LinkedIn)

What is an Agile Company?

What is an Agile Company?

2016 11 01 head

Lately people often ask me what is an agile company. They have an understanding of agile software development techniques such as Scrum, Lean, Kanban, eXtreme Programming but no clue how a company could be agile.

One way to approach this theme is to implement concepts of the above mentioned agile approaches. These are the pillars for iterative incremental improvements.

Scrum Pillars

Transparency
you trust all collaborators to provide their best when working on the product. They need full access to all relevant information: quality, progress, budget, people and financial data. They use the data to select the best alternatives to reach the company’s goals
Inspection
you cherish a failure culture because inspections will find weak points. You need a culture where people are never blamed. One consequence is that you eliminate management by objectives, and bonuses in your company
Adaptation
you are ready to learn better approaches and truly believe there is a better solution. Processes change regularly bottom-up, company’s budget can only be a rolling indicative budget, and planning has to be continuously updated

You can also pursue desirable attitudes or values of agile persons

Scrum Values

Commitment
as an employee you want to win, you are engaged. Every manager truly believes you work best to achieve the goals. If not, this manager made a mistake by hiring or keeping you
Courage
you are ready to say “I am wrong” because being wrong opens the door for improvement
Focus
you daily optimise the customers’ value and thrive for excellence. As a developer you do not work on projects, you work on excellent products. You provide value to your customers
Openness
you seek better ideas and ways of doing. Openness requires transparency and respect.
Respect
you truly respect customers, team members, collaborators, persons. In your team I will not hear any disrespectful comments about people

Modern Agile

The modern agile movement defines guiding principles

Make people awesome
Dedicated collaborators want to give their best. Give them a purpose, a goal, empower them and let them climb the mountain. Thriving persons need purpose, excellence and autonomy. They are happier, healthier and more productive.
Deliver value continuously
Maintain sustainable pace and stability, all divisions of the organisation should focus on customer value. At the end of the day successful products are the only reason you get your paycheck.
Make safety a prerequisite
People do their best in their daily work, you believe in McGregor Y theory. Trust them and support them to become excellent. You shall develop self-awareness to increase safety.
Experiment and learn rapidly
you make mistakes, you learn and increase continuously the delivered value. You have a learning culture, you inspect and adapt. You can react to new opportunities faster than competitors.

Self Assessment

The above concepts are comprehensible. How do you know if you are moving in the right direction. A few concrete tests help you find out if you are an agile department or company.

  • Be agile instead of do agile. Practice the above attitudes and do not just follow a checklist
  • Nobody micro-manages in your company
  • Feel accountable instead of be accountable. You want to improve, and your company as a natural part of the daily work.
  • Compliment every collaborator you are working with at least once a week instead of evaluating weaknesses and criticising people. o you lead by example?
  • No management by objectives - MBO - or bonuses are established in your company
  • Every collaborator has access to all company data, every collaborator can request process and tool changes. We favour Individuals and interactions over tools and processes
  • You want to be excellent in your work. You have a purpose and autonomy in your daily work
  • Team members take the decision to hire or to fire collaborators, not the department responsible or the human resources group. Think about collaborators selecting their leaders, about managers being servants, about information available to all collaborators
  • Can you say the most important one word Sorry, the most important two words Thank you, the most important three words I was wrong and the most important four words Can I help you? at least three times a week?

I truly believe that we all want a fulfilling job which improves our world. I cannot understand other reasons to spend 40 hours and more per week for something less valuable. Take the above principles and apply them to your daily work. There are universal values to establish a working atmosphere you are proud of.

I agree with all of you to desire a fulfilling job is only true if you earn enough money to pay your monthly bills.

Food for Thoughts

These ideas are not new. You can delve in empirical evidence and discussions in books written by business management professors, CEO, and passionate agile advocates. Below a list of mind openers (available as Amazon ebooks):

  • Reinventing organisations: A guide to creating organisations inspired by the next stage of human consciousness by Frederic Laloux,
  • Accelerate: Building strategy agility for a fast moving world by John P. Kotter,
  • Beyond budgeting: How managers can break free from the annual performance trap;
  • The Leader’s Dilemma: How to build an empowered and adaptive organisation without losing control; both books by Jeremy Hope,
  • Holacracy: the new management system for a rapidly changing world by Brian J. Robertson,
  • Deliver Happiness: A path to profits, passion and purpose by Tony Hsieh,
  • The Lean Startup: How today’s entrepreneurs use continuous innovation to create radically successful businesses by Eric Ries,
  • Lean Novels
    • The Lean Manager: A novel of lean transformation;
    • Lead with Respect: A novel of lean practice;
    • The Gold Mine: A novel of lean turnaround; all three books by Freddy Balle,
  • The Lean Mindset: Ask the right questions by Mary Poppendieck,
  • Social Intelligence: The new science of human relationships, by Daniel Goleman
  • Management 3.0: Leading agile developers, developing agile leaders by Jurgen Appelo,
  • The Fifth Discipline: The art and practice of the learning organisation by Peter M. Senge,
  • Fearless Change: Patterns for introducing new ideas; More Fearless Change: Strategies for making your ideas happen; both books by Linda Rising,
  • Excellence Novels
    • Build to Last: Successful habits of visionary companies;
    • Good to Great: Why some companies make the leap… and others don’t;
    • How the Mighty Fall: And why some companies never give in; all three books by Jim Collins,
  • Google re:work blog.

(this post was also published on LinkedIn)

Git Branches for the Impatient

Git Branches for the Impatient

2016 07 01 head

You are working in a small collocated development team and decided to use branches to implement new features or fix errors. Here the cookbook to create, edit, merge and delete local and remote branches in Git (version 2.x). Git branches have two important qualities.

  • A branch is like an idea. Once you implemented the idea, feature or fix you just merge back to trunk and delete the branch,
  • The history of the branch commits is still visible upon deletion of the branch.

You should use meaningful names for your branch name and associated commit messages. Put the ticket number into the branch name and messages for future searches.

The described approach is optimal for small teams. The approach is compatible to pull requests if you introduce such a workflow later. You do not need pull requests when you are working collocated . I prefer pair programming and pair check-in sessions against the trunk.

For a short introduction how to start using Git in software projects see the blog Git Local Repositories for the Impatient.

Create the Branch

Create new branch feat_#42 locally

git checkout -b feat_#42

Create the remote branch with the same name and initiate tracking, assuming your remote uses the standard default name origin.

git push -u origin feat_#42

Work on the Branch

Add your changes and commit them regularly.

git commit -a -m “commit message describing activities for feat_#42“

Upon running the unit tests locally, push the changes to repository

git push

Now you can test the branch from the central repository and deploy it to your continuous integration pipeline environment.

Merge the Branch

Switch to master and synchronize with your remote repository, the -p parameter means --prune

git checkout master
git fetch --all -p
git pull

Merge to master. The option --no-ff will always keep branch information.

git merge --no-ff feat_#42

Or if you want a single commit for the complete branch.

git merge —squash —no-ff feat_#42

Push the changes.

git push

For advanced users you can first rebase your branch and squash superfluous commits before merging the branch back to trunk.

Delete the Branch

Delete the remote branch (also git branch -dr origin/feat#42_).

git push origin --delete feat_#42

Delete the local branch.

git branch -d feat_#42

You are done. Now you are ready to implement the next feature.

View local and remote Branches

If you want to view branches use the following commands for the local branches.

git branch
git branch --no-merged

If you want to view remote branches.

git branch -r
git branch -r --no-merged

Checkout remote Branch.

The -p parameter means --prune

git fetch --all -p
git checkout #feat_42

More Information

You can find a lot of information on Stack Overflow. Beware when reading the answers on Stack Overflow that Git commands have changed over time. Select new posts to find the best answers.

The nifty-gritty details can be found in the official Git documentation.

Warning:

Beware that for example gitolite does not support special characters such as # in branch names. Use them only in the commit messages.

These same characters work in bitbucket.

The Version of the Scrum Guide 2011

The Version of the Scrum Guide 2011

2016 06 01 head

The new version of the Scrum Guide written by Ken Schwaber and Jeff Sutherland is available since July 2011. The older version was published in May 2009. I found quite interesting the precisions about the Scrum Master responsibility. I have often discussions with the customers and companies I coach about the responsibility of the Scrum Master.

Who is responsible for the introduction of the Scrum approach in the company?

Often people state that the management should spread Scrum in the company. Ken and Jeff have a clear idea who is in charge of this.

The Scrum Master serves the organization in several ways, including:

  • Leading and coaching the organization in its Scrum adoption;
  • Planning Scrum implementations within the organization;
  • Helping employees and stakeholders understand and enact Scrum and empirical product development;
  • Causing change that increases the productivity of the Scrum Team; and,
  • Working with other Scrum Masters to increase the effectiveness of the application of Scrum in the organization.

The Scrum Master is a main driver for the introduction of Scrum in all departments and levels in the company. He needs indeed senior management support but the Scrum Master daily job is to spread Scrum in the company.

Who should coach the product owner to write better backlog and clarify the vision?

The Scrum Master serves the Product Owner in several ways, including:

  • Finding techniques for effective Product Backlog management;
  • Clearly communicating vision, goals, and Product Backlog items to the Development Team;
  • Teaching the Development Team to create clear and concise Product Backlog items;
  • Understanding long-term product planning in an empirical environment;
  • Understanding and practicing agility; and,
  • Facilitating Scrum events as requested or needed.

The Scrum Master should also coach the product owner and support him in the long term planning, the communication of the vision and backlog management. The vision and goals of the product should always contain an overall planning frame. During the product empirical evidence will trigger revision of the milestones and product planning.

What is the meaning of done and a piece of potentially shippable product?

When the Product Backlog item or an Increment is described as “Done”, everyone must understand what “Done” means. Although this varies significantly per Scrum Team, members must have a shared understanding of what it means for work to be complete, to ensure transparency. This is the Definition of Done for the Scrum Team and is used to assess when work is complete on the product Increment.

Development Teams deliver an Increment of product functionality every Sprint. This Increment is working product, so a Product Owner may choose to immediately release it. Each Increment is additive to all prior Increments and thoroughly tested, ensuring that all Increments work together.

As Scrum Teams mature, it is expected that their Definition of Done will expand to include more stringent criteria for higher quality.

The Increment is the sum of all the Product Backlog items completed during a Sprint and all previous Sprints. At the end of a Sprint, the new Increment must be ““Done””, which means it must be in useable condition and meet the Scrum Team’s Definition of “Done”.

It must be in useable condition regardless of whether the Product Owner decides to actually release it.

So at the end of sprint all stories realized since the start of the product must be correct. Therefore you need automated acceptance tests to guaranty their correctness. The Scrum team must guaranty that at the end of each sprint all functions of the product work as specified and accepted by the product owner in the current and previous sprint reviews.

Higher quality is reached through extreme programming approach: test driven development, clean code, pair programming, pair check-in, coding dojo, static analysis tools.

Does Scrum consider long-term planning?

Yes it is a major activity performed by the Scrum Master and Product Owner. See the above point about the Scrum Master coaching the Product Owner. See if anyone states that long-term planning is not part of the Scrum theory you should challenge them and give them the Scrum Guide to read. What can be changed during a Sprint? Often customers have heated discussions what can be changed during a sprint. Here a clear statement.

During the Sprint: * No changes are made that would affect the Sprint Goal; * Development Team composition and quality goals remain constant; and, * Scope may be clarified and re-negotiated between the Product Owner and Development Team as more is learned.

What is measured in a burn down chart?

Scrum does not consider the time spent working on Sprint Backlog Items. The work remaining and date are the only variables of interest.

The only relevant information is the amount of work still open and when this work will be completed. This is true for sprint, release and whole backlog burn-down chart. I am glad that these precisions were added to the new version of the Scrum guide. I simplify my daily work. I can now simply refer the official Scrum Guide to convince people how aspects of Scrum should be understood. The older version of the Scrum Guide is still relevant because it contains hints and best practices no more part of the new version.

Scrum Master is a full-time Role

Scrum Master is a full-time Role

2016 05 01 head

I am very happy we discuss in depth the role, responsibilities and activities of a good Scrum Master. I fully agree with Scrum advocates that Scrum Master is a dedicated full-time job.

I am also convinced a Scrum Master can support multiple experienced Scrum teams. The Large Scale Scrum framework LeSS states a full-time Scrum Master supports 1 to 3 teams.

I worked with quite a few Scrum Masters, Product Owners, and Scrum teams. My experience shows it is more productive and effective when the Scrum Master works full-time as (see white paper of Barry Overeem)

  1. a servant leader
  2. a coach
  3. a facilitator
  4. a teacher
  5. a mentor
  6. an impediment remover
  7. a change agent
  8. a manager (please restrain yourself)

He should not implement stories. A Scrum Master is under huge stress being a development team member and having to balance their Scrum Master responsibilities against the commitment of the sprint goals.

So you shall have full-time Scrum Masters. More important is that you have real Scrum Masters, and not only Scrum administrators who organize meetings and manage the Scrum board and burn-down charts. Scrum has always been a hands-on approach, to be successful in this you need to have a passion for getting your hands dirty.

Find out if you are a real professional Scrum master. Do you perform each sprint the top things a Scrum Master usually forgets to focus on?

  1. Do Gemba. Redefine career paths, incentives, organization to be more Scrum focused,
  2. Identify with the product owner and the development team missing product backlog items,
  3. State team issues not being discussed because they are too uncomfortable,
  4. Find out appropriate balance between end-to-end system tests and unit tests,
  5. Play back the team’s progress against the proposed release plan,
  6. Integrate all tests into the continuous integration, delivery and deployment,
  7. Coach team members to understand the benefits of refactoring, removing waste and focus on quality,
  8. Teach Kaizen. Coach team to peer review and continuously improve towards perfection,
  9. Teach pair programming on a daily basis,
  10. Expand with the team and the organization the definition of done. Beware it can trigger an organizational change.

I hope the discussion what a successful Scrum master does stay active. Avoid agile coaches not being Scrum masters, they are not doing Gemba and loose contact with the knowledge workers.

Read our what you do not need to do.

It is a huge gain for the Agile community that people and organizations understand that a Scrum master is a professional coach, teacher, mentor, and change catalyst.

Books for Persons interested in Agile Approaches

Books for Persons interested in Agile Approaches

2016 04 01 head

Often I am told that it is difficult to use Agile/Scrum approaches for brown field projects or for big projects or for distributed projects or for in other situations.

Interestingly these persons also state that the same problem exist for RUP/OpenUP, Waterfall model DIN XT, Swiss variant called Hermes.

Agile approaches are currently the standard for new projects and are thought in technical universities.

Scrum is the main agile method and state of the industry approach for the implementation of software projects.

To open the discussion and bring the discussion back to objective arguments I often recommend the following books

Big Projects and Distributed Projects

  • Scaling Lean and Agile Development: Thinking and Organizational Tools for Large-Scale Scrum, Craig Larman and Bas Vodde, Addison-Wesley 2009 This book and the one below are a very extensive presentation about the challenges and trade-offs of big distributed agile projects. Don’t expect pre-cocked solutions but a tool set how to lead successfully such projects. Not always an easy reading but worth the effort
  • Practices for Scaling Lean and Agile Development: Large, Multi-side, and Offshore Product Development with Large-Scale Scrum, Craig Larman and Bas Vodde, Addison-Wesley 2010 See above for a review of the book
  • Agile Testing: A Practical Guide for Testers and Agile Teams, Lisa Crispin and Janet Gregory, Addison-Wesley 2009 The seminal reference book how to blend testing with agile projects. Lisa Crispin and Janet Gregory provides insights and a proven track how to guaranty agile quality assurance and agile testing. A must-read book for everyone seriously developing agile projects
  • Agile Estimating and Planning, Mike Cohn, Prentice-Hall 2006 The seminal reference about estimating agile projects. The experienced readers will understand why Mike Cohn used the words estimating and planning instead of estimates and plan.
  • The Enterprise and Scrum, Ken Schwaber, Microsoft Press 2007 The work of one of the Scrum founders how a company can adopt Scrum. You should read the standard works of Scrum to understand the concepts.
  • Agile Project Management with Scrum, Ken Schwaber, Microsoft Press 2003 The work of one of the Scrum founders how Scrum influence product organization and management. You should read the standard works of Scrum to understand the concepts.

Brown-Field Projects

  • Refactoring Improving the Design of Existing code, Martin Fowler, Addison-Wesley 1999 How to implement continuous improvement on the code level. If you are not refactoring you are not agile.
  • Working Effectively with Legacy Code, Michael C. Feathers, Prentice-Hall 2005 Most of us work on existing code bases. Based on the above statement you have to find a way to unit test and refactor legacy code if you want to be agile.

Extreme Programming

  • Test Driven Development by Example, Ken Beck, Addison-Wesley 2003 How can a programmer be sure his code is working before and after refactoring
  • Clean Code: A Handbook of Agile Software Craftsmanship, Robert C. Martin, Prentice Hall 2009 We are professional developers and uncle Bob shows how we should work
  • The Clean Coder: A Code of Conduct for Professional Programmers, Robert C. Martin, Prentice Hall 2011 Uncle Bob define what a professional developer is. I know quite a few developers shocked by his requirements

Agile and Lean Concepts

  • Lean Software Development, An Agile Toolkit, Mary & Tom Poppendieck, Addison-Wesley 2003 Classical work what lean development means
  • Implementing Lean Software Development: From Concept to Cash, Mary & Tom Poppendieck, Addison-Wesley 2007 The hand-ons how to implement lean approaches
  • Agile Product Management with Scrum: Creating Products that Customers Love, Roman Pichler, Addison-Wesley 2010 Requirement engineering done the agile way
  • Scrum and XP from the trenches: How we do Scrum, Henrik Knieberg, InfoQ 2007 Short book how to implement Scrum and XP in projects

Change Management

  • Fearless Change: Patterns for Introducing New Ideas, Mary Lynn Manns & Linda Rising, Addison-Wesley 2005 The introduction of Scrum and agile principles means change in the organization and the teams. Linda Rising shows how changes are introduced with success in existing organizations.

Books everybody should have read

  • Peopleware: Productive Projects and Teams, Tom DeMarco & Timothy Lister, Dorset House 1987 How to growth teams and lead successful projects
  • The Mythical Man-Month 2nd Edition, Frederick P. Brooks, Addison-Wesley 1995 The first version was published in 1975. The rules postulated by Brooks are still actual. Sad is that a lot of product leaders have no clue of these rules.
  • Becoming a Technical Leader: An organic Problem-Solving Approach, Gerald M. Weingartner, Dorset House 1986 How a gifted technical engineer can become a manager.
  • Slack: Getting past burnout, busywork, and the myth of total efficiency, Tom DeMarco, Broadway Books 2001 In one sentence, the tremendous difference between efficiency and effectivity.
  • The Dilbert Principle, Scott Adams, Harper Collins 1996 An entertainment presentation of all the mistakes companies are doing and still thinking they are smart
  • Death March: The complete Software Developer’s Guide to Surviving "Mission Impossible" Projects, Prentice Hall 1997 In my current coaching activities I still encounter departments where burnouts are common. Either these managers are criminals or so plain stupid that they cannot be held responsible. I am still unsure

I am curious about books you recommend for agile or other approaches. Just drop me an email or leave a comment.

PMI-ACP Certification

PMI-ACP Certification

2016 08 01 head

I learnt quite a few new acronyms and techniques when studying for the PMI-ACP certification program. In this post I collected these acronyms, some definition and the bibliography you should know before attending the examination. The main advantage of having it is that you no more need to argue with PMI certified product managers if agile is applicable to software projects.

The main interest of PMI-ACP certification is their collection of concepts taken from Agile, eXtreme Programming, Scrum and Lean.

It could worth your money to buy the book "The PMI-ACP Exam: How to pass on Your First Try", Andy Crowe, First Edition 2012. The book itself is not rocket science but it contains two hundreds questions which are similar to the ones you will encounter during the examination.

To get your 21 PDUs necessary for the examination you must attend a training course. I booked an online course at Simplilearn for a reasonable price. The content was reasonable. The test questions were good.

The renewal of the certification again requires PDUs. I took an online training in interesting areas such as coaching techniques, how to manage effective and efficient meetings or how to write good protocols. Some online videos of Mike Cohn also count for PDUs.

Chapter List

  1. PMI ACP eligibility, examination content
  2. Agile product management APM Framework, agile product lifecycle
    1. Envision,
    2. Speculate (During the Speculate phase, the product stories are delivered. This phase involves planning and delivering tested stories in a short iteration, constantly seeking to reduce the risk and uncertainty of the product.),
    3. Explore,
    4. Adapt,
    5. Close.
    6. Scope, Costs, Time
    7. The agile triangle: Value (releasable product), Quality (reliable, adaptable product), Constraints (cost, schedule, scope)
  3. People, Products and Processes
  4. Value, Quality and Constraints The five identified core risk areas for a product are: Intrinsic schedule flaw, Specification breakdown, Scope creep, Personnel loss and Productivity variation.
  5. Planning, monitoring, adapting
  6. Estimation in agile projects
  7. Communication on agile projects
  8. Analysis and design on agile projects
  9. Agile quality
  10. Soft skills for agile product leaders and negotiation
  11. Value based prioritization of requirements
  12. Managing risk on agile projects
  13. Metrics and charts on agile projects
  14. Agile value stream analysis
  15. Knowledge and skills
    1. Agile team members should be flexible and adaptable.

Bibliography

The examination reference list of PMI ACP is (the titles in italics are also part of the CAT reference list)

Agile Retrospectives: Making Good Teams Great, Esther Derby, Diana Larsen
  1. Agile Software Development: The Cooperative Game – 2nd Edition, Alistair Cockburn
  2. The Software Project Manager’s Bridge to Agility, Michele Sliger, Stacia Broderick
  3. Coaching Agile Teams, Lyssa Adkins
  4. Agile Project Management: Creating Innovative Products – 2nd Edition, Jim Highsmith
  5. Becoming Agile: …​in an imperfect world, Greg Smith, Ahmed Sidky
  6. Agile Estimating and Planning , Mike Cohn
  7. The Art of Agile Development, James Shore
  8. User Stories Applied: For Agile Software Development , Mike Cohn
  9. Agile Project Management with Scrum, Ken Schwaber
  10. Lean-Agile Software Development: Achieving Enterprise Agility, Alan Shalloway, Guy Beaver, James R

It is interesting to compare the above list with the reference list of the Certified Agile Tester CAT

  1. Agile Testing – A Practical Guide for Testers and Agile Teams by Lisa Crispin and Janet Gregory
  2. Succeeding with Agile: Software Development Using Scrum by Mike Cohn
  3. Agile and Iterative Development: A Manager’s Guide by Craig Larman
  4. Agile Project Management with Scrum by Ken Schwaber
  5. Agile Retrospectives: Making Good Teams Great by Esther Derby and Diana Larsen
  6. User Stories Applied: For Agile Software Development by Mike Cohn
  7. The Deadline by Tom de Marco
  8. Peopleware: Productive Projects and Teams by Tom de Marco & Timothy Lister
  9. Kanban by David J. Anderson
  10. eXtreme Programming explained: Embrace Change by Kent Beck

Terminology and Acronyms

The more terms and acronyms you know, the easier the examination will be.

2016 08 01 active listening

  • Active Listening
  • ARCS - Attention Relevance Confidence Satisfaction - relevant for motivational theory and process for systematic motivational design
  • Affinity Estimation - e.g. story points or teeshirt sizes -. The Affinity Estimating exercise is best conducted on Product Backlogs larger than 20 items. It is best when you have at least 40 items which allows for groupings to easily become apparent.
  • ATDD Acceptance Test Driven Development - Discuss, Distill, Develop, and Demo - see also Behaviour Driven Development BDD
  • Boundary, Authority, Role and Task BART
  • CD Continuous Deployment (CD as Continuous Delivery is not part of the examination)
  • Cumulative Flow Diagrams CFD
  • CI Continuous Integration: multistage integration is unning additional tests for performance, load or stability
  • Collaboration versus Coordination: Work Together versus Share Information
  • Cycle Time = Flow Time
  • DEEP Detailed Appropriately, Estimable, Emergent, Prioritised *Disaggregation: split story into smaller stories
  • DRY Don’t Repeat Yourself
  • EI Emotional Intelligence
  • EISA Emotional Intelligence Skills Assessment Perceiving, Managing, Decision Making, Achieving, Influencing
  • EQ Emotional Intelligence Quotient
  • Error-feedback ratio: is the number of new defects injected when fixing existing defects. Several years ago, Jerry Weinberg conducted studies on error-feedback ratio and found that a 20% difference in feedback ratio leads to an 88% difference in completion time (bad enough), but the next 10% increase leads to a 112% increase.
  • Earned Value Management EVM - this is standard PMI theory -
    • PV (Planned Value) = BAC (Budget At Completion) \* Planned Percentage Completed Budget Cost of Work Scheduled
    • AC (Actual Cost) - Budget Cost of Work Performed
    • EV (Earned Value) = BAC (Budget At Completion) \* Actual Percentage Completed - Sum (PV[Completed]) from start until current
    • CPI (Cost Performance Index) = EV / AC (Actual Cost) indicates if we are under or other budget
    • SPI (Schedule Performance Index) = PV / AC indicates if we are early or late
    • ETC (Cost Required) = (BAC - EV) / CPI - This metric is the forecast amount to complete the remaining work -
    • EAC (Forecast Cost for the total planned work) = BAC / CPI = AC + ETC
  • Five Level of Conflicts
    1. Problem to Solve (Good Teams) → Collaboration, consensus Collaboration- Seeking a win-win situation. Consensus- Learning where every team member’s head is with regard to the issue and, in time, arriving at a decision everyone can back.
    2. Disagreement → Negotiate, support
    3. Contest → Accommodate
    4. Crusade → Shuttle between parties
    5. World War → Protect to avoid injuries
  • JBGE Just Barely Good Enough
  • INVEST Independent Negotiable Valuable Estimable Small (Sized appropriately) Testable
  • Internal Rate of Return IRR, the higher the better. Internal Rate of Return (IRR) is used to express the return on product in % terms when comparing two different cash flow streams.
  • JIT Just In Time
  • Kano Model: Must have, Linear feature, Delight
    • Threshold or basic attributes are must have attributes otherwise the product is incomplete. Threshold features are those that must be present in the product for it to be successful. They are often referred to as must-have features.
    • Performance attributes are linear, the more the better
    • Excitement attributes are delights
  • MMF Minimally Marketable Feature
  • MoSCoW Must, Should, Could, Won’t
  • Net Present Value NPV → FV = PV * (1+i)^n, FV is future value, PV is present value, n is the number of periods/years, you can interpret as the higher the better
  • Payback Period, you can interpret as the lower the better
  • PESTLE Political, Environmental, Societal, Technological, Legal, Economical
  • PMI-ACP Project Management Institute Agile Certified Practitioner
  • Product Owner: Committed, Responsible, Authorized, Collaborative, and Knowledgeable
  • Project management methods
  • Relative Weighting Method
  • Retrospective: Set the stage, Gather data, Generate insights, Decide what to do, Close the retrospective
  • Risk
    • Risk Board
    • Risk Exposure (Risk Sensus) → Risk Probability * Risk Cost = Risk Exposure
    • Risk Management Process: Identify, Assess, Respond, Review of risks
    • Strategies: Avoid, Mitigate, Transfer, Accept
  • ROI Return On Investment (Benefits - Costs) / Costs in percent. The higher the better
  • ROTI Return On Time Invested (done in 60 seconds)
    • 0 = "I’d have been better off making a Starbucks run. Complete waste of time" or Lost Principle: No Benefit Received for Time Invested Break-Even
    • 1 = "You really should have let me stay at my desk and code"
    • 2 = "This was an OK meeting. About as valuable as if I’d been coding" or Received Benefit Equal to Time Invested High Return on Investment
    • 3 = "Surprisingly, this was more valuable than if I’d been writing code"
    • 4 = "Wow, this meeting saved me tons of time. Thank goodness I didn’t skip it to code" or Received Benefit Greater than Time Invested
  • RUP Rational Unified Process: Inception, Elaboration, Construction, Transition phases
  • Staging: The process of defining and prioritizing the nonfunctional requirements for scaling is called staging. Staging occurs prior to the start of the first sprint and takes just one day. During this day, the nonfunctional scaling requirements for this particular product are determined and placed in the Product Backlog.
  • Shu Ha Ri: can be considered as concentric circles, with Shu within Ha, and both Shu and Ha within Ri. The fundamental techniques and knowledge do not change.
    • "protect", "obey" — traditional wisdom — learning fundamentals, techniques, heuristics, proverbs
    • "detach", "digress" — breaking with tradition — detachment from the illusions of self
    • "leave", "separate" — transcendence — there are no techniques or proverbs, all moves are natural. Becoming one with spirit alone without clinging to forms; transcending the physical
  • Scrum
    • pillars: Transparency, Inspection, Adaptation
    • Scrum of Scrums = Meta Scrum
  • SDLC System Development Lifecycle
  • Servant Leadership
  • SIP Software In Progress
  • SMART - Specific Measurable Attainable Relevant Time-bound -
  • TFD Test First Development
  • Test Driven Development TDD
  • Extreme Programming XP
  • Wideband Delphi
  • Wave: Wave is the Product Planning structure with Medium range time frame (3 months) with story level capability and capability commitment. Waves, or milestones, are intermediate points, usually from one month to three months apart. Waves can have both a product management and a technical function. From a product management perspective, they provide a chance to review progress and make adjustments.
  • Work Breakdown Structure WBS

Below some additional definitions

Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.

Norm Kerth's Prime Directive (should be used in review and retrospective)
  • Story Points versus Ideal Days - and Elapsed Days -
  • Crystal Clear requires
    • the following properties:
      • Frequent delivery of usable code to users
      • Reflective improvement
      • Osmotic communication preferably by being co-located
    • Crystal Clear additionally includes these optional properties:
      • Personal safety
      • Focus
      • Easy access to expert users
      • Automated tests, configuration management, and frequent integration
  • Agile Coach Failure Modes: Spy, Seagull, Opinionated, Admin, Hub, Butterfly, Expert
  • Root-causing a defect or testing the feasibility of an algorithm or a third party solution is an example of a spike
  • Measure: ascertain the size, amount, or degree of (something) by using an instrument or device marked in standard units or by comparing it with an object of known size
  • Dysfunctional teams pyramid: absence of trust, fear of conflict, lack of commitment, avoidance of accountability, inattention to results
  • According to DeMarco, Fragmented knowledge workers may look busy but a lot of their business is just thrashing. The minimum cost penalty is 15%
  • The declaration milestone is a verbal notice from one person to another, or to multiple people, that a milestone was reached.
  • The three coach styles are Teaching, Coaching, and Advising.
  • Quantity of function is, scope, measured in terms of user stories, use cases, requirements, or features (depending on a particular situation). In software, these may be measured ultimately as objects, modules, classes, or lines of code.

Questions

  • Which of the following correctly defines the team members themselves managing assignment of the day-to-day tasks required to deliver stories at the end of each iteration? Workload Management
  • Which of the following technique can be used to apply to bring down the Lost Opportunity Cost within your team? Team Collocation
  • When should the Product Owner provide feedback on the work results? Just in time reviews
  • Who is the most appropriate person to monitor all the risks in an Agile product? The product manager
  • Which of the following is false about Velocity? Velocity cannot correct Estimation errors
  • Which of the following is NOT the skill for Agile coaches for facilitating change? Reaching agreement
  • Which tool combines the estimation techniques of expert opinion, disaggregation and analogy? Planning poker combines expert opinion, analogy, and disaggregation into an enjoyable approach to estimating that results in quick but reliable estimates.
  • Which of the following metrics can be BEST standardized across teams? Business case realization -The PMO can help the teams to enable timely decision making by standardizing the metrics.-
  • Which is the communication technique where you repeat back a summary of what the other person just said to you to confirm understanding? Reflective Listening is a communication technique where you repeat back a summary of what the other person just said to you to confirm understanding. Another benefit in this situation is that having the person hear their own ideas in another person’s voice/words may make it easier for them to be objective.
  • Adaptation depends upon understanding a wide range of information, including an assessment of the product’s progress, technical risks, the requirements evolution, and ongoing competitive market analysis. Which are the areas where every team needs to constantly evaluate and make appropriate adaptations ? Product Value, Product quality, Product status, Team performance -Every team needs to constantly evaluate and make appropriate adaptations in the following four areas - Product Value, Product quality, Product status, Team performance-
  • A standard for measuring or evaluating something. Metric - A metric is a standard for measuring or evaluating something.-
  • What BEST describes the characteristics of a Learner at Level 3 or in the Fluent stage of learning? Learners in the Fluent stage are experts -The Stage 3 Learner is at the stage of mastery. He is able to figure out the end effect of any procedure and to make his way to that end.-
  • Scrum uses the sashimi technique to require that every slice of functionality created by the developers be complete. All the requirements gathering and analysis, design work, coding, testing, and documentation that constitute a complete product are required to be completed in every Sprint and demonstrated in the Sprint increment of functionality. Sprints are kept short enough that the stakeholders don’t lose interest in the product before the Sprints are completed. And stakeholders can see that they have an opportunity to redirect the product at the start of every Sprint to optimize the value they derive from the product. At the end of every Sprint, stakeholders see new functionality. Models, requirements, and internal artefacts might be of use to the developers, but they are never shown to the stakeholders.
  • Feature X has a value of 12 and the total value of all features is 35. If the feature is estimated to cost 56%, what is the priority of this feature using relative weighting? Correct Answer is B. The priority of the feature is determined by dividing the relative value by the cost %. Hence the answer = (12/35)/(0.56) = 0.61.
  • The 100-Point Method was originally developed by Dean Leffingwell and Don Widrig for use cases and is used for prioritization as well. It is a voting scheme where each stakeholder is given 100 points that he or she can use for voting in favour of the most important requirements. How they distribute the 100 points is up to them: 20 here, 10 there or even all 100 on a single requirement if that is their sole priority.
  • When a team member approaches the Coach with a complaint about another team member, what conflict resolution technique should the Coach use? Three-step intervention path Every team needs to constantly evaluate and make appropriate adaptations in the following four areas: Product value, Product quality, Team performance, Project status.
  • Normative methodologies are based on solutions or sequences of steps known to work for the discipline. Electrical and other building codes in house wiring are examples. In software development, one would include state diagram verification in this category.
  • Iterative development means that we build a partial version of a product and then expand that version through successive short time periods of development followed by reviews and adaptations. Feature-based delivery means that the engineering team builds features of the final product or, particularly with industrial products. At least a close representation of the final product (such as a simulation model). Iterations are constrained to produce a result within a certain period of time—a time box (as short as 1–4 weeks for software). Time boxes force closure; they force us to make something concrete, often before we are quite ready. Incremental development means that we build these products such that they could be deployed at the end of one or more of the iterations.
  • Forecasting the financial value of a theme is the responsibility of the product owner, but it is a responsibility shared with all other team members—programmers, testers, analysts, product managers, and so on.
  • Decision Framing focuses majorly on, Decision framing focuses on who gets involved in the decision process. Managers who make decisions without input from subordinates and peers make poor decisions. Engineers who make decisions without input from managers and peers make poor decisions. Who makes the decision is less important than getting the right people involved in the decision process.
  • Which of the following charts shows the total number of story points completed through the end of each iteration? Cumulative story point burn-down chart
  • During a critical problem solving, you can ask probing questions, use active and reflective listening, Lead to an answer but one should avoid injecting their own ideas.

I wish you success for your certification.

Why I use a MacBookPro and OS X

Why I use a MacBookPro and OS X

2016 03 01 head

As a young developer I loved Linux, compiled new kernels during evening sessions and struggled days to have the correct drivers for the graphic card and communication components of my notebook.

I grew older and decided to enjoy my evenings and weekends. Surely I would prefer to have no virus, trojan and other evils on my workstation. So I went to macOS and Apple notebooks without regrets.

Gains

The major gains upon migrating to OS X are

  • No virus scanner is necessary, the speed-up during complex programming and development activities are tremendous,
  • No trouble when updating OS, updates are automatic and often no new start is required,
  • Unix Command Line and Tools are available in the console. Homebrew or MacPorts projects provide all known and less-known utilities and programs available under Linux,
  • Long Lived Notebook is still working and looking nice after five years of daily usage. The performance is more than adequate for software development with Java 8/9/10/11 stack and C++.

Daily Development

The tools I really enjoy and use on a daily basis for software development - mainly Java - are

  • IntelliJ IDEA IDE,
  • Atlassian Cloud applications - BitBucket, HipChat, CI pipeline -,
  • Homebrew as package manager for utilities,
  • Docker as container manager,
  • VirtualBox when I need a full-fledged virtual machine.

Daily Work

I use these tools to perform administrative work are

  • LibreOffice,
  • Google Business for team work in the cloud using collaborative tools,
  • Apple Mail Client with GPG plugin for PGP and S/MIME secure email,
  • A local Swiss article Banana for accounting and VAT reports for the federal government. I bought it as soon as the company stopped requesting higher prices for OS X than for the other platforms.

LibreOffice completely replaced Microsoft Office suite. I stopped using OpenOffice after the strange behavior of Oracle with the application.

I never really regretted leaving Linux or Ubuntu behind me.

Agile Trends Switzerland 2013

2016 10 01 head

What are the main hurdles to introduce agile approaches in Swiss companies

  • Introducing agile company-wide is a cultural change process. Such a change takes time and sometimes hurts,
  • Without commitment of senior management, the initiative will fail,
  • Doubt that lean or agile works for big projects, doubt you can convince your collaborators and middle management, doubt your customers agree with lean,
  • At the end what matters is collaborator purpose, customer satisfaction, business value.

SwissQ has published a "SwissQ Agile Trends & Benchmarks Switzerland 2013". The study can be downloaded as PDF from their website. See Agile Trends Switzerland 2012.

Below some of the findings of the study.

Major Hurdles When Introducing Agile Approaches

The number left is the value for 2013, the number in parentheses is the value found in the study for year 2012. Bold items are new in the 2013 study and have no 2012 values.

  • 72% (95%): Capabilities to handle organizational changes and the culture
  • 53% (-%): Difficulty how to handle the lost of control - from management perspective -
  • 28% (37%): Availability of skilled collaborators in the area of agile approaches
  • 25% (34%): Projects are too big or too complex
  • 23% (39%): Overall resistance against changes
  • 21% (-%): Scalability
  • 21% (28%): Missing support of senior management
  • 14% (25%): Company wide introduction of agile methods, doubts agile approaches scale
  • 14% (31%): Resistance for the internal or external customer
  • 11% (23%): No resources or time for sustainable changes
  • 4% ( 9%): Costs reasons

The trend is a clear move toward agile approaches. Companies seem less reluctant to introduce agile approaches. Major opponents such as senior managers or internal and external customers more and more acknowledge the success of agile approaches. The two first identified hurdles still reflect the difficulty of changes at company level. They are also the main reasons why Scrum introduction fails in Swiss companies for my experience. Too often companies revert to some Waterfall/Hermes process or worst to a so-called internal ad-hoc method.

We wrote in previous blogs that introducing Scrum in the development department or in the whole company is a change process. The coaches should be trained in change management and have experiences with resistance to changes.

The Scrum Master is of tremendous importance. You need a skilled and enthusiastic Scrum Master; avoid Scrum Administrator. Ken and Jeff have a clear idea who is in charge of this.

The Scrum Master serves the organization in several ways, including:

  • Leading and coaching the organization in its Scrum adoption;
  • Planning Scrum implementations within the organization;
  • Helping employees and stakeholders understand and enact Scrum and empirical product development;
  • Causing change that increases the productivity of the Scrum Team; and,
  • Working with other Scrum Masters to increase the effectiveness of the application of Scrum in the organization.

The Scrum Master is a main driver for the introduction of Scrum in all departments and levels in the company. He indeed needs senior management support, but the Scrum Master daily job is to spread Scrum in the company.

Swiss IT CIO Agenda 2013

Computerworld Switzerland has published an article about the main reasons why projects are late or cancelled in Switzerland in the April 2013 CIO Agenda edition. The study uses data from Computerworld and Experteer. As an excerpt I gave you the four major identified failure reasons.

  1. Changes during the product (62% Computer, 48% Experteer) Agile and Scrum welcomes changes in the product. It is one of the four mantra of the Agile Manifesto. All agile processes are optimised to support changes.
  2. Unclear product definition form the business stakeholders (50% Computer, 43% Experteer) Scrum requests a clearly stated vision, roadmap and story map of the product. The Scrum board is a radiator to communicate the product vision, goals, and motivations to all interested parties.
  3. Insufficient product management and controlling (40% Computer, 4847 Experteer) Scrum team, product owner, and Scrum master have clear responsibilities for product management. At the end of each sprint we have an objective and explicit control point.
  4. Insufficient communication between involved persons (36% Computer, 45% Experteer) Scrum framework enforces clear, transparent and regular communication within the team and with all stakeholders.

In one sentence, be agile, use Scrum and eXtreme Programming and the major four above problems are handled in a professional and consistent way.

Agile Trends Switzerland 2012

2016 09 01 head

What are the main hurdles to introduce agile approaches in Swiss companies.

  • Introducing agile company-wide is a cultural change process. Such a change takes time and sometimes hurts,
  • Without commitment of senior management, the initiative will fail,
  • Doubt that lean or agile works, doubt you can find agile experts, doubt that your projects can be realised with agile approaches,
  • At the end what matters is collaborator purpose, customer satisfaction, business value.

SwissQ has published a "SwissQ Agile Trends & Benchmarks Switzerland 2012". The study can be downloaded as PDF from their website.

Below some findings of the study.

Major Hurdles When Introducing Agile Approaches

  • 95%: Capabilities to handle organizational changes and the culture
  • 39%: Overall resistance against changes
  • 37%: Availability of skilled collaborators in the area of agile approaches
  • 34%: Projects are too big or too complex
  • 31%: Resistance for the internal or external customer
  • 28%: Missing support of senior management
  • 25%: Doubts that agile approaches do scale
  • 23%: No resources or time for sustainable changes
  • 9%: Costs reasons

We wrote on blogs that introducing Scrum in the development department or in the whole company is a change process. The coaches should be trained in change management and have experiences with resistance to changes.

The above points in italics are all related to change processes. You can handle two thirds of the hurdles with adequate change management training and a clear change product definition. Do not try to introduce Scrum at company level without such an approach.

Main Reasons Why Agile Projects are Cancelled

  • 52%: Missing experience with agile approaches
  • 45%: Company culture clash with agile principles
  • 41%: External pressure to use traditional approaches (Agile 51%, Waterfall 40%, Iterative 22%, RUP 16%, and HERMES 12%)
  • The used agile approaches are:
    • Scrum 84.5%,
    • Kanban 16.9%,
    • Own Approach 15.5%,
    • XP 14.1%,
    • Agile Unified Process 11.3%,
    • Others 9.9%,
    • SCRUMBAN 8.5%,
    • Feature Driven Development FDD 0%
  • 38%: Missing support from senior management
  • 37%: Missing or insufficient training and coaching
  • 35%: Missing communication between departments
  • 23%: Team does not want to learn new approach

We wrote in previous Scrum master is a full time job that the Scrum Master is of tremendous importance. You need a skilled and enthusiastic Scrum Master; avoid Scrum Administrator. The new version of the Scrum guide clearly stated who is responsible for the introduction of the Scrum approach in the company? Often people state that the management should spread Scrum in the company. Ken and Jeff have a clear idea who is in charge of this.

The Scrum Master serves the organization in several ways, including:

  • Leading and coaching the organization in its Scrum adoption;
  • Planning Scrum implementations within the organization;
  • Helping employees and stakeholders understand and enact Scrum and empirical product development;
  • Causing change that increases the productivity of the Scrum Team; and,
  • Working with other Scrum Masters to increase the effectiveness of the application of Scrum in the organization.

So the Scrum Master is a main driver for the introduction of Scrum in all departments and levels in the company. He needs indeed senior management support but the Scrum Master daily job is to spread Scrum in the company.

Do Agile Methods - Scrum - Motivate your Team?

Do Agile Methods - Scrum - Motivate your Team?

2016 02 01 head

Well my current experience is that the answer is a sounding YES.

Developers love to develop software using Scrum. The motivation and positive energy just explodes. People talk together, build as a team better products and enjoy the daily activities.

Stakeholders and in particular customers or users embraces the advantages they get.

A whisper can still be heard:

Yes, it is not without consequences for the organization.

Say your team is developing software products with Scrum. We have the following roles:

  • the stakeholders
  • the developers (Development Team)
  • the Scrum master (SM)
  • the product owner (PO)

The Scrum Team is the composition of the development team, Scrum master, and the product owner.

Stakeholders

Stakeholders love Scrum. Every two weeks they have a new application version they can use, test, and get feedback from all users. This is the dream of every key account manager.

Because a new release is available every two weeks the stakeholders should also provide feedback every two weeks. They sometimes hate the pressure of agile and Scrum.

Developers Team

Agile methods - Scrum, XP - are fun for all developers. Suddenly everyone can see your code, so at the beginning you as a developer must cope with different views how good code should be written. Suddenly you must master Test Driven Development TDD, mocking, unit test, legible documents. But, once you have it, you start to experience the "flow" and enjoy the atmosphere and to deliver at the end of each sprint a new software.

Later you realize that Scrum says the team is responsible. You must acknowledge that the product success is also your success. Each design decision is now also driven by ROI considerations. If you master these responsibilities you suddenly discover the world of engineering. It is creating great products with an optimal balance of external quality and costs without ever jeopardizing the quality of the core. You can never again deny that you as developer are in charge.

Scrum Master

Scrum master is cool with a team enjoying Scrum and the flow of success. You just have to realize you are not a master but the enabler of a team of talented engineer.

So you need to discover your "the lonely cowboy riding in the sun" song why this job is so rewarding.

Product Owner

As a product owner you control the budget of the product, the priorities and at the end of each iteration you can change all priorities and goals. Cool isn’t it. But wait as a product owner I am responsible for the budget and product success and each time a member of the team asks me something I must have an answer ready.

Scrum is cool and we enjoy it. And Scrum makes everything transparent and fast. Every two weeks a new version is delivered. This version must be evaluated, accepted or a list of improvements must be provided in the next days. This rhythm is fast, really fast. Once you are up to this speed you will enjoy the flow.

Cautious Words

Middle management often struggles with the agile approach at the beginning. See for example the discussion about manager role in an agile environment in the LeSS approach. Be assured the results are worth the effort.

Reading experience report or digging in the findings of Kotter you realize success or failure goes through how you handle middle management.

Minimizing Undone Work when Working with Regulatory Departments

Minimizing Undone Work when Working with Regulatory Departments

2016 01 01 head

The Scrum and agile mantra is to have a "ready to ship" product each time a sprint is completed. You must avoid any incomplete activities at any price.

Incomplete activities are also called Undone Work; they are technical debts in your software, hindering its delivery to customers.

Regulatory and traditional quality insurance departments requests all kinds of documents such as review, test results, risk analysis matrix. They want insurance that their view of quality is fulfilled.

Scrum sees quality similar to the lean production approach. Quality is build into the product and you do not need any documents proving you have fulfilled this goal. Your sole goal is to optimize the process to produce the best products, not to repair them after production.

It is the sole responsibility of the Scrum team to guaranty quality. But Scrum teams also learnt you cannot win against big corporations. So you have to define solutions satisfying the QA departments with minimal overhead and almost no undone work. We have experience working with quality insurance and TUV or FDA driven regulatory departments. We use the following approaches to handle the needs of the regulatory departments, seen as stakeholders in the Scrum terminology.

  • Use static analysis tools such as PMD, Checkstyle, Findbugs to create quality gate reports for your QA department. Thousands of checks and hundreds of pages of reports will satisfy any QA department. Plugins exist for Eclipse and continuous integration servers.
  • Use review plugin such as Eclipse Jupiter to create these still mandatory review reports on the source code. These reports are useless in an agile team working with techniques such as refactoring and pair programming. But they are still mandatory for the majority of traditional QA departments. The plugin allows you the generate nice looking reports and to implement the findings at the same time.
  • Use Test Driven Development approach and create a lot of unit tests. By slightly extending the logging features of JUnit and by using code coverage tools you can create reports showing which source code was tested. Add some annotations to your test cases, and you have full traceability to your stories and associated software requirements. With a small amount of work you can generate thousands of pages of information and provide the PDF documents to the regulatory department. These documents are also TUV and FDA compliant.
  • Use Behavior Driven Development approach and create acceptance criteria for all your stories. Using the reporting features of easyb you can create acceptance test reports for all your stories. It generates hundreds of pages of information and provide the PDF documents to the regulatory department. These documents are also TUV and FDA compliant
  • Use continuous integration servers to run all the above measures and generate the documentation at the end of each sprint or release.
  • Link you version control system with your continuous server, issue tracking system and unit tests. So you can generate release notes stating the list of closed issues with the associated source code and unit tests verifying the correctness of the change.

Similar tools are available in the .NET and C++ community. T You can easily satisfy your QA and regulatory stakeholders with a lot of reports nobody will ever read. A real Scrum team corrects weaknesses during the development process and always deliver a software of the highest quality. The reports can only show the inherent quality of the software and are not worth the paper they are printed on. But you satisfy powerful stakeholder groups and can more freely work on the most important goal. Deliver on time and in budget the best software the customer wants to have.