This the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

2018

How Healthy is Your Product? - Product Strategy Check

How Healthy is Your Product? - Product Strategy Check

2018 12 01 head

You are developing your product using agile and lean approaches.

How can you check your approach and distill improvements? A health check of your product and your development approach is certainly a good solution.

This post is the third of a set of articles identifying health checks with different focus. We will identify strengths, potential weaknesses and hopefully find room for improvement.

What is your product strategy and how do you involve your customers and stakeholders.

Rules

The patient solely decides if a health check shall be performed. The effort and depth of a health check is defined together with the patient. The findings are considered confidential.

A short check with a feedback workshop presenting the insights requires around two work days.

A deeper and more intensive check requests more effort, between one and four weeks. Upon completion you can also hire a coach to implement selected findings and improve durably the health of your application. The findings are often clear and the measures straightforward. The real work is to implement consistently the measures.

I have the same personal challenge when trying to loose weight or improve my running form. The long term implementation is the crux.

Product Strategy Checks

Vision of the product
You can communicate the vision on your product in an elevator pitch and have it hanging in the team workplace.
Roadmap of the product
You have a visible roadmap describing the major releases for the next 12 to 18 months. Define the right planning horizon for your article.
Release plan of the product
You have a release plan for the next 6 to 9 months.
Story map
You have a story map grouping the stories and epics to reflect your release plan.
Customer involvement
You have customer interaction in every sprint. Not only during the review event, but also during the planning meeting and architecture discussion.
Review with customer involvement
Realized stories are discussed during the review and demonstration meeting and timely feedback is provided to allow backlog refinement before the next planning meeting. A mature organization has acceptance criteria for each story and automated verification of the criteria at any time.
Early and regular delivery of an increment of the product
You get feedback on the delivered and installed product increment. This goal is only achievable if your organization has implemented {url-articles}2018/blog-2018-11-01[Delivery Pipeline Check],
Risk List
You have a documented risk lists with description of risks, probability of occurrence and cost per event, and mitigation measures,
Solution architecture
You have an emergent solution architecture with prototype and support the team to refine the architecture proposal and to implement it in the solution. Do not forget an agile product needs an agile architecture. See also the LeSS approach for creating an agile architecture.
Non functional requirements
You have identified your non functional requirements and constraints and have fitness measures in place. The measures guarantee the continuous achievement of the constraints - often defined as -ility requirements -.

See the {url-articles}2018/blog-2018-11-01[Delivery Pipeline Check] to enable your organization to provide realtime feedback how well the above constraints are correctly implemented in the product.

Posts in the Health Check Series

SAFE 4.6: Improvements Direly Needed

SAFE 4.6: Improvements Direly Needed

2018 11 02 head

It is official. The power engine of SAFe 4.6 vehicle is Team and Technical Agility. View the official presentation and more important download the slides. It is one of their five Lean Enterprise competencies and one of their four core values. SAFe finally recognizes the tremendous importance of technical proficiency and craftsmanship.

They advocate two findings. First you need team agility, how you apply Scrum or Kanban. In other words being agile instead of doing agile. Second you need technical agility, how you develop a quality product.

I challenge you. If you are following SAFe, talk with your teams and checks how many of their official good practices are you using on a daily basis.

I have seen SAFe introductions in Swiss, German and Czech companies. Attention was set on the structure of the release train and a lot of effort was spent on the various ceremonies, meetings and artifacts. But how to produce quality software and apply technical excellence was sadly always an afterthought. And more than once a senior SAFe coach just publicly stated that the SAFe core value x is not so important for a successful introduction. Quite a denial approach of his own methodology I think!

Be honest with yourself and your teams, either you implement the techniques described in https://v46.scaledagileframework .com/team-and-technical-agility/[Technical Agility] and Build-In Quality chapters of SAFe or you are not agile and have no SAFe power engine. It is called cargo cult.

A good starting point are the five dimensions of build-in quality described in SAFe 4.6. And yes now SAFe acknowledged the tremendous importance of a continuous delivery pipeline. Experts agree you can apply DevOps only if you have continuous integration, delivery and deployment.

Once you realize where your organization stands remember that build-in quality is one of tour core values of SAFe. That I mean is walk the talk.

And please read some Internet blogs and books about LeSS, Software Craftsmanship, Extreme Programming, TDD, ATTD, clean code. Most of these concepts were defined and published in the last millennium. It is time your teams finally adopt them and produce high quality, maintainable and cost-effective products. This is the whole concept of the lean movement. And I welcome the fact that SAFe now explicitly mention Cost of Delay.

One of the strengths of SAFe is their clear statement that an agile organization transformation is only successful with lean-agile leadership. All C-level managers are trained in lean and agile, they advocates these approaches and are visible champions. Therefore all managers shall be advocate technical excellence on a daily basis.

If not the seminal work of John Kotter clearly states the transformation will fail. You will have a bunch of Scrum teams in your development department, but have none of the expected synergies at product or company level.

Personally I prefer the LeSS approach, but I respect your choice - SAFe, Nexus, or Scrum@Scale - as long as technical excellence is a key aspect of your agile strategy. Embrace technical agility and do not let your train of teams produce mediocre software every two weeks.

Employ professional software craftsmen and deliver quality products. Follow your SAFe approach, respect one of the four core values and one of the five core competencies of your method.

See also my blogs {url-articles}2018/blog-2018-04-01[Pragmatic Craftsmanship], {url-articles}2018/blog-2018-06-01[Software Quality Graal], and How healthy is your Product? article series for information about technical excellence.

The other big improvement is that the PI horizon is now limited to 8 or 12 weeks. No more semesterly or hopefully quarterly cadence.

It still does no make sense to plan for three months and at the same time advocate DevOps and design thinking.

DevOps has a time horizon of days, it is waste to have huge quarterly PI gatherings and plans just to discard them after the next DevOps finding. You should probably just update your PI roadmap.

Call for Action

My call for action is fewer meetings and paper artifacts and more technical excellence. You want to sell excellent and well-designed products, not mediocre software.

How Healthy is Your Product? - Delivery Pipeline Check

How Healthy is Your Product? - Delivery Pipeline Check

2018 11 01 head

You are developing your product using agile and lean approaches.

How can you check your approach and distill improvements? A health check of your product and your development approach is certainly a good solution.

This post is the second of a set of products identifying health checks with different focus. We will identify strengths, potential weaknesses and hopefully find room for improvement.

How you build and deliver your software product has direct implication on your liquidity and cash flow. Improvements in your delivery pipeline have direct impact on your finances. === Rules

The patient solely decides if a health check shall be performed. The effort and depth of a health check is defined together with the patient. The findings are considered confidential.

A short check with a feedback workshop presenting the insights requires around two work days.

A deeper and more intensive check requests more effort, between one and four weeks. Upon completion, you can also hire a coach to implement selected findings and improve durably the health of your application. The findings are often clear and the measures straightforward. The real work is to implement consistently the measures.

I have the same personal challenge when trying to loose weight or improve my running form. The long term implementation is the crux.

Delivery Pipeline Checks

How reproducible, fast and economic is your product deliver pipeline? At the end of the day you deliver regularly a reproducible and trackable product version to your customers.

One nice aspect about delivery pipeline checks is that a whole set of tools and methods emerged with the DevOps movement. You still need an expert to tune the diagnostic to your company specific needs but the approach is well documented.

Below a list of capabilities we have build-up over time for delivery pipeline checks. Use common sense and strategic goals to identify the objectives you want to measure. Expertise is needed to define values to measure if the source code of your product reaches the selected objectives.

  1. Continuous integration pipeline with quality checks and automatic tests,
  2. Continuous delivery pipeline packaging the same binary to all your environments,
  3. Continuous deployment if adequate for your solution, certainly it is adequate for the test and integration stages,
  4. Automatic source code quality checks and mandatory quality gates,
  5. Automatic unit tests, integration tests and end to end tests,
  6. Golden Trunk in Git,
  7. No branches or only short-lived branches policy,
  8. Zero bug policy - eliminates the change management processes and board for errors -,
  9. Traceability of specifications to delivered features and product versions,
  10. Semantic versioning and API documentation,
  11. Configuration and infrastructure as managed artifacts,
  12. Generation of documentation part of the delivery pipeline,
  13. Monitoring and alarming,
  14. DevOps pipeline analysis provides another look at your processes
    1. Code development and Review, Continuous integration and build status,
    2. Continuous testings and business risks feedback,
    3. Artifact repository and pre-deployment staging,
    4. Release automation including change management and release approvals,
    5. Infrastructure configuration and management and infrastructure as code,
    6. Applications performance monitoring and end-user experience.

You should know how often you can deliver your product without failures and track your lead time between fixes. The overall goal is faster time to market and improved delivery frequency.

The overall features and processes can be implemented through your development organization is less than a year. Stop evaluating, start implementing!

Continuous Integration Practices

Check your process to see which continuous practices you are really mastering

  • Maintain a single source repository - no long-lived branches
  • Automate the build - no manual inputs, no manual configuration, automatically triggered
  • Make your build self testing - TDD, ATDD, static metrics, semantic checks
  • Everyone commits to trunk every day
  • Every commit should build trunk on an integration machine
  • Fix broken build immediately - lean red button stopping production, everybody helps fix the broken build Keep the build fast - We are talking of minutes not hours
  • Test in an integration staging area which is a clone of the production environment - the goal is to have docker images, virtual network, and in-memory database to be fast
  • Make it easy for anyone to get the latest packaged release
  • Everyone can see the build status - transparency, information radiator
  • Automate deployment

Posts in the Health Check Series

How Should You handle Definition of Done?

How Should You handle Definition of Done?

2018 10 02 head

An agile team is sole responsible for the internal quality of the product they build and maintain.

And the developers shall be accountable to produce the expected quality and optimize long terms goals of the organization. They provide the answer to the seminal question

Did we build it right?

Definition of Done DoD is a major building block to answer this question with a sounding yes.

Definition of Done

Scrum and LeSS have a stringent discussion concerning Definition of Done. The key question is

_Considering our current context and capability, what activities can be completed each Sprint?

This subset is the initial Definition of Done.

A Definition of Done is weak when it is a small subset and strong when it is almost equals to Potentially Shippable.

In huger organizations the development teams discuss their context and select the subset of the activities that all teams think they realistically can do during the Sprint. This is their initial Definition of Done. The teams that can do more will expand this product Definition of Done within their members.

The difference between the Definition of Done and Potentially Shippable is referred to as Undone Work.

Potentially Shippable - Definition of Done = Undone Work

The Sprint is planned according to the Definition of Done and thus the Undone Work is excluded. It is planned to be left undone.

The terms Potentially Shippable and Definition of Done are often not used consistently. To clarify the terms:

Potentially Shippable
All activities that must be performed before the product can be shipped._
Definition of Done
An agreement between the teams and their Product Owner on which activities are performed inside the Sprint. A Definition of Done is perfect when it equals to Potentially Shippable. Teams strive to improve towards a ideal Definition of Done.
Undone Work
The difference between the Definition of Done and Potentially Shippable. When the Definition of Done is perfect then there is no Undone Work. If this isn’t the case then the organization has to decide, first how do we deal with the Undone Work, and second how do we improve so that there is less Undone Work in the future.
Unfinished, not finished, or not done—Work
that was planned in a Sprint but was not completed. This is often confused with Undone Work.
Unfinished
is work that the team planned for but did not finish whereas Undone Work was never even planned for. When a team has work that was not finished then they ought to feel anxious and discuss improvement actions during their retrospective.

Teams should never leave work-in-progress at the end of the Sprint and “carry over” to the next one. This causes a lack of transparency and reduces scope flexibility. If they forecast too much work, they need to remove complete items which they haven’t started yet.

Items of a Definition of Done

  • A delivery standard as defined by the team,
  • It contains all requirements to get a user story into production,
  • The fitness for use is evident (enough value was built to justify releasing?),
  • The external quality is verified (we have built the right thing?),
  • The internal quality is verified, you wrote unit tests – and all tests are green,
  • The code is checked in, if necessary the branch was closed and deleted,
  • The code review was completed,
  • All improvements from the code review were implemented,
  • All existing unit tests remain green,
  • The acceptance tests were verified by the development team,
  • All integration tests were passed,
  • and the Status of “done” was confirmed by the product owner.

The following criteria are often overlooked and also need to be checked:

  • Was the technical documentation updated?
  • Was the user documentation updated?
  • Was the user documentation localized?
  • The localization for the application is done.
  • The localization testing is done.
  • The marketing input is done.
  • The legal documents are done.
  • The deployment and migration scripts are available.

The extension of definition of done to deliver a potentially shippable product has often significant and profound impact on the structure of the organization and its processes. Experienced agile coaches use the definition of done as an instrument for organizational changes.

When the undone work is slowly removed from the delivery process it triggers structural and process changes. For example the separate quality department responsible for the final tests is dissolved and their expertise is integrated in the development teams.

See also the blog Pragmatic Craftsmanship for a discussion of build-in quality.

How Healthy is Your Product? - Source Code Check

How Healthy is Your Product? - Source Code Check

2018 10 01 head

You are developing your product using agile and lean approaches.

How can you check your approach and distill improvements? A health check of your product and your development approach is certainly a good solution.

This post is the first of a set of articles identifying health checks with different focus. We will identify strengths, potential weaknesses and hopefully find room for improvement.

The initial examination shall always be the source code. Your code base is a key asset of your company.

Rules

The patient solely decides if a health check shall be performed. The effort and depth of a health check is defined together with the patient. The findings are considered confidential.

A short check with a feedback workshop presenting the insights requires around two work days.

A deeper and more intensive check requests more effort, between one and four weeks. Upon completion you can also hire a coach to implement selected findings and improve durably the health of your application. The findings are often clear and the measures straightforward. The real work is to implement consistently the measures.

I have the same personal challenge when trying to loose weight or improve my running form. The long term implementation is the crux.

Source Code Checks

How clean, legible and maintainable is your source code? Often it is best to start your health evaluation with your product source code. At the end of the day it is compiled source code you deliver to all your customers to work with.

One nice aspect about source code checks is that a whole set of tools are available on the market. You still need an expert to provide the diagnostic but the gathering of data can be automated and quite inexpensive to gather.

Below a list of capabilities we have build-up over time for source code checks. Use common sense and strategic goals to identify the objectives you want to measure. Expertise is needed to define values to measure if the source code of your product reaches the selected objectives.

  1. Java 11/10/9/8/7/6 code and how to migrate to a current version and use the new Java features,
  2. Multi-threaded, parallel programming and reactive systems,
  3. Functional and object oriented detailed design approaches,
  4. Test driven development TDD and build-in quality - gradle or maven -,
  5. Acceptance test driven development ATDD and user acceptance,
  6. Clean code approach and detection of code smells,
  7. Static analysis tools to measure the quality attributes of source code,
  8. Compilation and test automation,
  9. Software craftsmanship approach, values and techniques,
  10. Domain driven development and micro services,
  11. Database models and open data,
  12. Software architecture and Enterprise architecture,
  13. CI and CD - continuous integration, continuous delivery, continuous deployment,
  14. Static quality assurance approaches for quick diagnostics,
  15. Development tools to improve quality and productivity,
  16. User Experience and UI design.

My blogs Pragmatic Craftsmanship and How to Reach the Software Quality Graal provides insights why source code quality is of tremendous importance for any company.

And the blog {url-articles}2018/blog-2018-05-01[SonarLint] for the Impatient would be a pragmatic first measure to measure and improve your source code health.

Any product should have a battery of measures deployed to continuously monitor the source code and detects trends over time.

Posts in the Health Check Series

Introduce Agile in A Company

Introduce Agile in A Company: Shu Ha Ri

2018 09 01 head

First teams have successfully used Scrum to develop applications. The news spread out and everybody wants to apply Scrum. It is time to introduce Enterprise wide Agile with Scrum or Kanban or XP.

Your Company wants to introduce Enterprise Agility in their product development, quality assurance and sales departments. The company as a whole shall transition to an agile organization and all teams shall adopt Scrum.

Try the Shu Ha Ri approach for such a change process.

Based on our exhaustive experience we recommend starting with a set of workshops with the goal to define a tailored approach to adapt the structure and processes of the company and introduce enterprise agile.

These workshops provide the platform to identify the company’s needs and strategic goals, and gather support from all participants. Try the open space format to involve all collaborators and identify their needs.

Below we provide an approach which must be later tailored to the specific needs of the Company during the above mentioned workshops. In particular the organizational needs for feature oriented planning and development of a portfolio of applications should be considered.

Approach

We work with the Shu Ha Ri agile approach when introducing enterprise agile in the Company. All collaborators will progress from the novice level through the proficient level before reaching the expert level. You can also use the agile fluency model if you prefer.

This model also maps very well to the “Awareness – Desire – Knowledge – Ability – Reinforcement “or ADFAR model. The Company has already reached the awareness and desire level because they want to introduce enterprise agile. Below we describe the activities for the development organization consisting of the software development, product management and quality insurance departments, and the activities at company level.

Development Organization

Shu - Novice Level - „Learn the Rules“

The duration of the novice level is at least six months. The major activities are to align the collaborators and provide the needed knowledge through training and hands-on exercises. The extensive training is aligned with experiences and industrial best practices.

All collaborators shall attend an introduction to agile and Scrum approaches. All technical collaborators shall attend the Scrum - Master or Developer - training. Each group of technical collaborators will learn specific techniques and skills necessary to work effectively in an agile product environment.

The agile coaches have two roles. First they facilitate and support the teams to implement all meetings, artifacts, and rules of the Scrum approach. Second they train the team members how to use the learnt techniques such as clean code, pair programming, pair check-in, agile architecture, requirements elicitation, test driven development, agile quality assurance, acceptance test driven development, continuous integration, deployment and delivery.

After the initial training the coaches ensure that this knowledge and ground rules are applied in daily activities.

DeveloperBusiness Analyst / POSM / PL / Line ManagerQuality Manager / TesterOthers
Agile and Scrum Introduction(1 day)
Scrum Master (2 days)
Product Owner (2 days)
  • agile and clean developer
  • TDD
  • ATTD
  • Agile Development Game
  • DVCS – Git –
  • Legacy code refactoring
  • Clean Code
  • Agile architecture and patterns such a inversion of Control
  • Agile Requirement Management
  • Lean Startup
  • Lean UX
  • Design Thinking
  • ATDD
  • Agile Leadership
  • Advanced Lean
  • Agile Quality
  • Soft skills – coaching, mentoring -
  • Agile Quality Management
  • ATDD
  • Move them to a Scrum team
?
~ 20 days~ 10 days8 days8 days1 day

We recommend

  1. An Agile (eXtreme Programming techniques) and Scrum coach responsible of two to three Scrum teams.
  2. Coach of coaches for the growth of internal agile and Scrum coaches for the company.

Ha – Proficient Level - “Bend the Rules”

The duration of the “proficient level” will be up to two years. The goal is to reinforce the learnt techniques and become proficient in using them. T he motto is

Most difficulties in Scrum don’t arise from learning the Scrum framework. Rather they arise in applying the framework on an existing organizational structure and culture.

This requires hands-on experienced guidance, and not just at the team level, but at the leadership and organizational level. See below the discussion about enterprise transformation.

We recommend

  1. An Agile (eXtreme Programming techniques) and Scrum coach responsible of four to five Scrum teams.
  2. Two coaches of coaches for the growth of internal Scrum coaches for the company.
  3. Start to blend the team coaching with the company wide changes. See below the chapter “Organization Transformation”.
  4. Workshops could be organized for selected products or teams to solve identified major impediments.

Ri – Expert Level - “Break the Rules”

The “expert level” will be reached earliest after two years. It is reasonable to state that not all collaborators will reach this level of expertise. The goal is to transfer ownership of the agile approaches to Company collaborators. The company will take over the responsibility to reinforce agile and Scrum activities and resolve newly identified impediments in products or at company level. The motto is

*Doing* agile is easy. Being agile is hard.

We recommend

  1. An Agile and Scrum coach responsible for six to eight Scrum teams. The internal coaches should start taking responsibilities
  2. A center of competence for the growth of internal Scrum coaches for the company.

Company Transformation

While teams implement Scrum, managers and leaders enable them. We work on the cultural, organization, and process level to enable agility in the company.

Workshops with key managers will

  • Discuss the fact that the standard competencies of agile leadership and coaching – mentoring, facilitating, problem-solving, and conflict navigating – are essential to agile adoption, but simply not enough to sustain and grow agility within an organization. We find that the unique blend of combining senior internal leaders with experienced agile coaches provides an unprecedented learning environment for all.
  • Understand that most organizations adopt agile "Outside-In" - that means they start with a process change and expose organizational impediments. The problem is that most of the organizational impediments are driven from cultural values deep within the organization. the process changes rarely stick,
  • Identify an "Inside-Out" approach - that means we will start with the company culture. Just as understanding the personality of a person is a key to working effectively with that person, understanding the culture of an organization is critical to its success in adopting, and more importantly sustaining, agility.
  • Explore a number of organizational systems and structures which enable (or impede) agility at an organizational level. This approach sets the priority first to the culture, second to the structure and third to the processes of the company

We recommend

  1. Workshop with key managers as described above,
  2. One Coach at C-level,
  3. Transform the company one product line after the other, delay the broad deployment to have time to learn and improve,
  4. Study LeSS as a valuable approach. We strongly support LeSS as an ideal candidate for company wide agile approach,
  5. Implement selected company-wide measures during the second phase of the change management process when the development department moves to proficient level. Top managers must realize that cultural elements must first be modified before the structure and the processes of the company can be adapted,
  6. Be patient and do not loose your mojo.

Checklist

Here a checklist example to reflect why and how to transform into an agile organization

  • Answer the question, “Why move to Agile?” This is important, as the reasons for attempting such a fundamental change should be well understood from both a quantitative as well as qualitative standpoint,
  • Understand the current business culture. Change is hard and there will be champions as well as potential saboteurs of the changes to come,
  • Spend time on the organization structure to understand how it helps or hinders the move to agile and lean,
  • Involve all levels of the business, including top level ‘C’ executives. Their sponsorship and support will be important,
  • Don’t neglect mid-level management as their support is vital to the success of the transformation,
  • Create a roadmap with the explicit understanding that it will change over time,
  • Don’t attempt to change everything. Pick an area where a win will be evident and beneficial,
  • Prefer narrow and deep change to broad and shallow approach.

What does an Agile Coach?

What does an Agile Coach?

2018 08 01 head

An agile coach has three main areas of activity:

Foster Autonomy
Teach and support the team and individuals to be autonomous within the given boundaries of the organization and its processes.
Create Ground for Motivation
Help teams to maximize their autonomy, individual mastery and make sure the team has a purpose.
Provide Coaching
Support teams and individuals to identify potentials with their roles and within the agile organization and improve.

How Can You Do It?

Lateral Leadership
Mediating the team through conflicts and help them with making decisions and foster the team’s self-organization. After a few months a team shall have jelled.
Facilitation
Facilitate meetings for the team, including preparing, moderation and post processing for the agile process as well as the supporting agile organization. After a year of working with a team, the team members shall be empowered to run regularly all events without you attending.
Teaching
Teach teams agile practices and lateral leadership. A central area is technical excellence and software craftsmanship. Long lasting teaching shall concentrate on values: Scrum values and principles, Less principles, Agile manifesto and the associated 12 principles.
Impediments
Actively identify impediments and foster their removal. Consult the Scrum guide: You must select one impediment after each retrospective and add it as high priority item in the next Sprint backlog. This mechanism insure continuous improvement.

To ensure continuous improvement, it includes at least one high priority way in which the team works, identified in the previous Retrospective meeting.

Culture
Establish a culture of inspect and adapt for teams , create structure to establish a continuous improvement process - also meaning establish a tolerance to failure -, advocate transparency, enlighten the organization of the relation between autonomy and accountability.

Ask yourself how your organisation handles meritocracy.

One Approach to Coaching

Agile teams thrive with purpose, autonomy and mastery. One approach to coaching is to set your focus on

  1. Understand the domain: Talk with customers and users and understand their needs. Your teams should have business expertise,
  2. Master Technical Excellence: CI, CD, TDD, ATDD, pair programming, refactoring, lean UX, usability
  3. Use Process Excellence and apply Scrum, XP, LeSS by the book:
    1. Discuss the core values of commitment, focus, openness, respect, and courage,
    2. Present again the agile manifesto and the twelve principles
    3. Show the organization why commitment in the planning was replaced by forecast in the official Scrum guide,
    4. Require a vision, a roadmap, a release plan with a story map and a refined backlog for each product and team,
    5. Tell why an impediment resolution backlog item shall be part of each iteration,
    6. Teach the importance of Definition of Done for the internal quality of the product. And explain why Definition of Ready is often an anti-pattern,
    7. Show why velocity is not a productivity metrics,
    8. Show the difference between product and output,
    9. Extend the agile toolbox with Design Thinking, Lean Factory, Lean UX, DevOps,
    10. Pair Scrum master and technical master to implement a craftsmanship path and achieve product quality,
    11. Explore Center of Competencies to nurture your product development group,

Once you are a master fell free to break the rules, experiment with new rules, approaches. Define hypotheses and verify them empirically.

If you are coaching software development groups acknowledge you shall possess profound understanding of technical excellence and software development practices. I never heard of a soccer coach not intimately knowing how the game is played.

Architectural Decision Records ADR

Architectural Decision Records ADR

2018 07 01 head

Architecture Decision Record ADR is a compact and traceable documentation of all important decisions taken during the realization of a product. Use asciidoc or Markdown and Git to have an agile and lean documentation of your major decisions.

The huge advantage of Architecture Design Record ADR is the timeline of the decisions. It tremendously helps to understand the technical changes during the product lifecycle.

Please try architecture design records to document your key decisions when developing your software product. Feel free to identify better approaches

Definition

An Architectural Decision (AD) is a software design choice that addresses a functional or non-functional requirement that is architecturally significant. A decision can be documented in an architecture design record (ADR).

In each ADR file, write these sections:

Title
short present tense imperative phrase, less than 50 characters, like a git commit message.
Status
proposed, accepted, rejected, deprecated, superseded
Context
what is the issue that we’re seeing that is motivating this decision or change.
Decision
what is the change that we’re actually proposing or doing.
Consequences
what becomes easier or more difficult to do because of this change.

I use asciidoc format to write all our architecture design records. You can add to the same git repository as the product, visualize differences and if desired generate a nice looking document. And my IDE IntelliJ IDEA has a nice plugin for this format.

Before using ADR I documented the decision changes as footnote in a Microsoft Word document. The ADR approach is way more legible and easier to trace with the help of git repositories.

Title

These documents have names that are short noun phrases. For example, "ADR 1: Deployment on Ruby on Rails 3.0.10" or "ADR 9: LDAP for Multiple tenant Integration"

Context

This section describes the forces at play, including technological, political, social, and product local. These forces are probably in tension, and should be called out as such. The language in this section is value-neutral. It is simply describing facts.

Decision

This section describes our response to these forces. It is stated in full sentences, with active voice. "We will …​"

Status

A decision may be "proposed" if the product stakeholders haven’t agreed with it yet, or "accepted" once it is agreed. If a later ADR changes or reverses a decision, it may be marked as "deprecated" or "superseded" with a reference to its replacement(s).

The status and the graph of related decisions can be tracked and visualize using tools. You can also track your decisions without any tools, a regular product has often less than hundred decisions. And you can always write a small parse utility to extract your dependency graph.

Consequences

This section describes the resulting context, after applying the decision. All consequences should be listed here, not just the "positive" ones. A particular decision may have positive, negative, and neutral consequences, but all of them affect the team and product in the future.

Tips and Tricks

The whole document describing an ADR should be one or two pages long. We will write each ADR as if it is a conversation with a future developer. This requires good writing style, with full sentences organized into paragraphs. Bullets are acceptable only for visual style, not as an excuse for writing sentence fragments (Bullets kill people, even PowerPoint bullets).

The cost of undocumented decisions is hard to measure, but the effects usually include duplicated efforts. Other engineers try to solve the same problems or competing solutions (two third-party libraries that do the same thing).

The approach is very similar to how you would document a software pattern.

Believe and respect is the prime directive.

Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand.

Norman Kerth

How to reach the Software Quality Graal?

2018 06 01 head

Lately I was asked how many bugs and smells detected through SonarQube were acceptable for a software product. And a Scrum master inquired which organizational changes are necessary to empower software developers to write good code. Speechless and dumbfounded I was unable to answer these questions.

After quite some time I realized these persons believe tools can measure the quality of software. But tools can only measure non-quality.

They tell you when you are doing it wrong, only your heart can tell you when you are doing it right.

But first that does it means to have a quality software product?

A high quality software product has no catastrophic failures, and seldom bad behavior.

It is maintainable and extendable with new technologies and new features.

The first part is called extrinsic quality and is why customers buy your product. The second part is called intrinsic quality and is why customers buy your product again and recommend it.

Criteria such as test code coverage, static analyzers, coding guidelines are just tools to detect missing quality. They help your team produce higher quality and nothing else.

I encounter organizations stating that zero defects as measured with SonarQube is the ultimate goal - and they find normal to have hundred of thousands of smells -. Time for a reality check.

Tools such as SonarQube, unit test coverage are just instruments to support your teams to detect known non-quality.

How to find the Quality Graal?

Below I provide a set of established, old techniques to eliminate non-quality. Most of them are around for at least twenty years. I encourage you to extend your practices with additional procedures based on your measurements and findings.

Extrinsic Quality - Product Quality

Is why you buy a specific car

  • Acceptance criteria and behavior test driven development ATDD - You insure through automatic tests the feature behave how your customer requested,
  • Defect driven development - You insure through automatic tests your customer will never encounter this exact error again,
  • Fast release and deployment cycle
    • You give your customer a way to detect bad behavior early and have an opportunity to ask for improvement. By fast we mean daily or perhaps weekly.

Intrinsic Quality - Technical Quality

Is why you can daily drive your car and not have to weekly repair it. And, later be able to add a charger for your mobile phone without spending a fortune.

  • Static code analysis - Known weaknesses are removed from your source code,
  • Test driven development TDD - You can change the code without changing the behavior or outputs, see also ATDD,
  • Clean code - You guarantee the source code is maintainable,
  • Clean architecture and domain driven development - You guarantee the product is extendable.

Accountability

The Scrum Team - Development Team, Product Owner and Scrum Master - are responsible and accountable for the intrinsic and extrinsic quality of the delivered products. Insufficient quality is always a collective failure.

Truly if the intrinsic quality of the source code is low, it is undeniably a failure. A failure of the development team and of each software developer writing the code; and a failure of the Scrum master holding and teaching agile and Scrum values.

When there is high pressure on delivery people have a tendency to pay less attention to processes and standards. That is one of the reasons why in Scrum we have the role of Scrum master responsible for understanding and enacting agile practices and rules.

And no physician will ever cut corners just because he is in a stressful situation. So please be a professional and read again the principles of the agile manifesto such as

Continuous attention to technical excellence and good design enhances agility.

Agile Manifesto

Developers please read the seminal book Clean Coder [2] from Robert Martin. As a professional you are responsible and accountable for the quality of the software you produce.

I find intolerable to hear software professionals stating they are not in charge of the intrinsic quality of the product they write. They cowardly try to push the responsibility to the internal quality assurance department or to the whole organization.

The above ideas are keys for Software Craftsmanship.

Kaizen - Small Steps bring First Successes

The first steps to improve your product are quite simple are around for at least twenty years.

  • Eliminate all compiler errors and warnings,
  • Use static checkers,
  • Activate the checkers of the IDE you are working with. For example, I am using IntelliJ IDEA and they have extensive checkers,
  • If using SonarQube, download {url-articles}/2018/blog-2018-05-01]SonarLint plugin and simply use it -. This tool is best of breed, do not argument and just use it -,
  • Write tests with JUnit5, Mockito, AssertJ, Cucumber.

I am ashamed to see software developers not implementing the above measures on a daily basis. I despair encountering Scrum masters who tolerate the situation. These techniques cost nothing, are installed in a few minutes and support incremental improvement of your source code. See also Why software developers do not need to ask for permission or forgiveness.

Your goal is simply zero warnings at the end of the spring. Later tighten it to zero warnings in the trunk of your git repository. Just do it - and stop arguing about the merits and costs -!

And if you do not agree with one of the warnings just disable the rule and document the reason in an architecture design record ADR.

Agile and Scrum are all about learning and continuous improvement. Please stop talking about quality and just start improving it! I do not want to hear any lame excuses why a software developer is not continuously doing the above activities.

I acknowledge I am emotionally involved and not fully objective. Lately I saw Java code of three different commercial products and I am still recovering from the missing quality.

Literature

  • [1] Clean Code, Robert C. Martin, 2009
  • [2] Clean Coder, Robert C. Martin, 2011
  • [3] Clean Architecture, Robert C. Martin, 2018
  • [4] Clean Agile, Robert C. Martin, 2020
  • [5] The pragmatic Programmer: Your Journey to Mastery (20th Anniversary Edition), Andrew Hunt & David Thomas, 2019

Agile Trends Switzerland 2018

2018 05 02 head

What are the main hurdles to introduce agile approaches in Swiss companies. The Swiss agile trends report was published for the last ten years and learn us the difficulties and successes of agile transformations.

SwissQ has published a "SwissQ Software Development Switzerland 2018", the tenth version of this report.

Introducing agile company-wide was and is a cultural change process. Such a change takes time and sometimes hurts. Too often organizations stop the process and revert to all command and control approaches with Waterfall product management.

Without commitment of senior management, the initiative will fail. In other words Walk the talk.

You must teach, coach, and support your collaborators through the transition. You need to believe in them and provide an environment where they can safely learn and experiment.

At the end what matters is collaborator purpose, customer satisfaction, and business value.

The study can be downloaded as PDF from their web site. See previous blogs for the results of years 2016, 2013 and 2012. Below some of the findings of the study.

Slower Adoption of Agile Approaches

The adoption of agile frameworks has markedly slowed down in Switzerland. And often hybrid methods based on Waterfall methodology are used. It is fascinating to see companies trying to merge a linear approach with highly iterative and incremental approaches. It shall be clear that success will only be elusive.

The data of the study reflect this fact through the abysmal collaborator satisfaction with these hybrid approaches (see chapter 13). Less than 25% of employees are satisfied with the development process of their company.

Major Hurdles When Introducing Agile Approaches

The list of hurdles for a successful introduction of agile approaches was updated.

69%
Culture change and you need senior management support
54%
Non adapted overall processes because local optimizations have limited effect
42%
Missing product overview because a company sells products
42%
Focus on business value because a company shall earn money
25%
Missing agile know-how of collaborators and you shall train your collaborators

We reached a major threshold. The majority of interviewed companies have more than 500 IT collaborators. These big companies are struggling to introduce agile approaches company-wide. Interestingly 75% of all companies plan to increase investment in the agile introduction and transition projects.

One interesting consequence is the company compiling the report is actively poaching potential agile candidates from these big laggard companies!

The majority of teams use Scrum or Kanban. The associated events and tools are used. The deeper values such as test driven development, acceptance test driven development, clean code or refactoring are often missing. The key is:

To be agile, not to do agile

We wrote in previous blogs that introducing Scrum in the development department or in the whole company is a change process. The coaches should be trained in change management and have experiences with resistance to changes. Our experience shows senior management are often not ready to move along the cultural change path.

Sonar Lint for the Impatient

Sonar Lint for the Impatient

2018 05 01 head

Remove issues before they reach customer sites. No quality problems shall reach your Git master branch.

What is Sonar?

SonarQube is a central server that processes full analyses (triggered by the various SonarQube Scanners). Its purpose is to give a 360° vision of the quality of your code base. For this, it analyzes all the source lines of your product on a regular basis.

It should be added that SonarQube also performs scans with well-known third party analyzers (SpotBugs - previously FindBugs -, Checkstyle, PMD) whereas SonarLint does not include those. I think the reason is a prioritization on performance and findBugs relying on java byte-code.

What is SonarLint?

SonarLint lives only in your IDE (IntelliJ, Eclipse or Visual Studio). Its purpose is to give instantaneous feedback as you type your code. For this, it concentrates on what code you are adding or updating.

Findings are displayed in the source code with information such as rule violation, rating of the rule and explanation. You have a window where all findings are displayed to provide an overview and navigation support. You are free to analyze a single file, a package, or the whole product; you can also exclude files or packages from the analysis.

Connect SonarLint to your preferred code quality application. It then uses the same code analyzers, rules and settings locally that are used on the server.

Issues marked as Won’t Fix or False Positive in SonarQube or SonarCloud disappear from your IDE.

Both SonarLint and SonarQube rely on the same static source code analyzers - most of them being written using SonarSource technology.

How to use It?

  1. the SonarLint in your favorite IDE
    1. configure the SonarCube instance to synchronize the rule set used in your product
  2. Configure the plugin to access your SonarCube instance
  3. Run it,
    1. See the messages in the next few seconds
    2. Jump to the source and correct the problem
    3. Iterate until all issues are solved
  4. Run your unit tests and acceptance tests - TDD and ATDD -
  5. Commit your changes

This approach guarantees you always release source code with less SonarCube issues, therefore SonarCube always show no findings for new code and for refactored code.

And you have made a step to the journey to become a professional software developer and achieve {url-articles}2018/blog-2018-04-01[craftsmanship].

Agile Introduction Success Criteria through the LeSS lens

Agile Introduction Success Criteria through the LeSS lens

2018 04 02 head

An agile organization is able to deliver or change direction at any time without additional cost.

More companies want to become truly agile and book an expensive journey to the holy land of agility. Often the travelers never reach their true destination.

The LeSS people have a clear set of conditions to increase the odds.

All my experiences confirm these conditions are essential for success.

LeSS Criteria

Use these criteria as a checklist to evaluate your current agile transformation

  • Educate everyone
    • Train all involved collaborators extensively, train them multiple times,
    • let them apply the learnt to their daily activities,
    • Select an experienced coach understanding organization, team, and technical excellence,
    • Plan for a long term engagement of the coach. The journey will take years, *Define your “product”
    • You need an elevator pitch describing your product family and why the customers is buying,
    • Identify your vision, your roadmap for the next three years, your releases for the next 12 months, and product backlog and boundaries of your product,
  • Define “done” - Definition of Done
    • Definition of done tells when your work is completed,
    • A strong definition of done supports organizational changes, see the LeSS metaphor of the Undone Department,
    • Check the team can implement the current definition of done or expand cross-functionality and training,
  • Have appropriately structured teams
    • A team is stable, long-lived, colocated, cross-functional and implement end to end features for the solution. Avoid component teams specialized for database, user interface or communication layer,
  • Team members are dedicated to the team. They do not work in other projects and do not report to a matrix structure,
  • Only the Product Owner provides work for the teams
    • The team is dedicated to implement the best solutions as defined by the product owner. No side activities are tolerated,
  • Keep product managers away from the teams
    • Project managers are not needed in an agile organization,
    • We realize and sells products, projects are irrelevant,
    • Nurture good managers to sustain the transformation.

Further Reading Read the LeSS Rules for ideas and wisdom how to improve your approach to agile product development.

As you found out now, I am convinced that LeSS approach is one of the best ways to truly transform into an agile organization. And less truly and deeply advocate technical excellence for all collaborators. You need an agile mindset and technical excellence to develop awesome products - see Software Craftsmanship blog.

A huge advantage of LeSS is that it is Scrum. You do not need to learn new methodologies, terminology or processes. Just apply all your hard earned learnings from your Scrum transformation.

Pragmatic Craftsmanship - Professional Software Developer

Pragmatic Craftsmanship - Professional Software Developer

2018 04 01 head

The last months I was often confronted with software products having insufficient quality.

Insufficient means late delivery, and few new features. The effort to correct errors in the field eats 30% to 50% away from the whole development budget. How can such a dreadful situation occur?

One root cause is the low quality of the source code and ignorance of basic coding design approaches. We are not talking about complex design activities or complex patterns; we are discussing basic approaches how to write correct and maintainable code.

Master Your Technology Stack

Most of the programmers have no formal training in the technology stack they are using daily. Their best friend is the Internet. And therefore many agile projects are now, steadily and iteratively producing mediocre software.

You shall be a proud software craftsman. You shall work as a professional and deliver professional results. I recommend for any Java developer

  • Have formal training in the Java language and library. For example you should consider a Java Programmer certification for the current JDK you are working daily with,
  • Learn the new features in Java e.g. Streams, Lambdas, Reactive Programming, LocalDate, Modules, packaging with jlink,
  • Read and understand Effective Java <<[effectivejava>> from Joshua Bloch,
  • Read and understand Clean Code [1], Clean Coder [2], Clean Architecture [3] and Clean Agile [4],
  • Read and understand Refactoring [9] from Martin Fowler,
  • And read Refactoring not on the backlog article from Ron Jeffries
  • Learn modern practices such as logging with slf4j, TDD with JUnit5, Mockito and AssertJ, ATDD with Cucumber,
  • Lean DevOps, continuous integration, continuous delivery, continuous deployment, monitoring,
  • Be aware of famous Java libraries such as Guava, Apache Commons,
  • Use SonarQube - and the SonarLint plugin for your preferred IDE - to catch well-known weaknesses in your source code and improve your coding skills,

A software developer not knowing the above items is almost certainly not a professional person, just a more or less talented amateur. See the blog of Mike Cohn the difference between aprofessional and an amateur for a similar point of view.

Improve Continuously You shall strive for mastery.

The feeling when you reached a certain mastery level is truly awesome. * Read a technical book each quarter, * Lean a new language or framework every two years, * Read books such as The pragmatic Programmer [5] * Know approaches such eXtreme Programming, and read the book <<[extremeprogramming>> * Work with Scrum, Kanban, LeSS -. Agile approaches are definitively state of the industry techniques -, * Master container approaches with Docker - containers are here to stay and will replace virtual machines -.

You shall write good enough software without errors and using best practices of the industry. If not consider changing your profession. See my blog post about technical debt discussing the consequences of missing mastery.

Software Craftsmanship

Software craftsmanship is an initiative to improve the professionalism of software developers. You do not have to agree with them, just look what they are doing.

For a detailed discussion read for example the book The Software Craftsman: Professionalism, Pragmatism, Pride by Sandro Mancuso.

Don’t discuss practices, discuss value

Their manifesto is

Craftsmanship is not enough to guarantee the success of a product but the lack of it can be the main cause of its failure.

Agile and Craftsmanship complement each other and both are necessary

Agile processes assumes technical excellence and a professional attitude

Software craftsmanship takes technical excellence and professionalism to a whole new level

Take everything you read with a grain of salt. Software craftsmanship

  • Is not a church, trying to convert all developers,
  • Is about leading by example and showing how we can be better,
  • Is about well written and designed code - and the customer stays in the center -,
  • Is about continuously delivering value not writing crap code.

Your goal shall to become a professional developer and reach mastery. How you do it is up to you.

Literature

  • [1] Clean Code, Robert C. Martin, 2009
  • [2] Clean Coder, Robert C. Martin, 2011
  • [3] Clean Architecture, Robert C. Martin, 2018
  • [4] Clean Agile, Robert C. Martin, 2020
  • [5] The pragmatic Programmer: Your Journey to Mastery (20th Anniversary Edition), Andrew Hunt & David Thomas, 2019
  • [6] eXtreme Programming Explained: Embrace Change, Ken Beck, 2005
  • [7] The Software Craftsman: Professionalism Pragmatism Pride, Sandro Mancuso, 2015
  • [8] Effective Java: Best Practices for the Java Platform (3rd Edition), Joshua Bloch, 2018
  • [9] Refactoring: Improving the Design of Existing Code (2nd Edition), Martin Fowler, 2018

An extensive list of books can be found under Books.

Why Do You Need Metrics and KPI in Agile Product Development?

Why Do You Need Metrics and KPI in Agile Product Development?

2018 03 03 head

Why do we define metrics and KPI?

First we want to find out how good the build-in quality of our software artifacts is. Second we want to improve the build-in quality through experiments and use the metrics to judge success of these experiments.

And the Key Performance Indicators - KPI - provide plausibility checks by qualifying the intrinsic quality of software artifacts against quantifiable strategic objectives.

Measurements shall be cheap to gather and as much as possible generated automatically through the tooling landscape.

We provide two sets of measurements to find out how supportive the development process is for quality and how to evaluate the quality of the artifacts.

Definitions

A metric is a measure or a combination of measures for quantitatively assessing or improving a product, a process or a team.

A key performance indicator is a metric that

  • is tied to a strategic objective,
  • have at least one defined time-bound target value - a number, a trend, a variation, etc. -

Therefore you must defined your strategic objectives before you can use KPIs.

Lean approaches have an imperative to reduce cycle-time, eliminate waste and tend to zero-defect products. The last one is a corollary because defects are waste. Please use this imperative to define your agile and lean KPIs.

If your tooling landscape does not support automatic gathering of a specific metric, you shall make it optional until you have solved your tooling weaknesses. Accurate and real-time automated measurements are of tremendous importance for acceptance and lasting success.

Process Aspects

The optimal lean software development process has a batch size of one, almost no waiting time, no superfluous stock and no quality issues.

Cycle Time of defects
It shows the effectivity to resolve non-quality in product development,
Cumulative Flow Diagram for defects and for stories/Backlog Items
It shows how effectively resources are allocated,
Sprint and PI burndown planned stories and effort
versus completed stories and effective effort trends show the stability and adaptability of the development process,
Planned velocity and effective velocity trends
It is an indicator for the maturity of the agile product organization.

This metrics shall be automatically provided through the issue and product backlog system. For example the “best of breed” JIRA tool has build-in reports for all the above metrics, so the tools you use should also have them.

Technical quality

A high quality software artifact has no code smell issues, no compiler warnings, always compile, and all tests are successfully processed.

  • Code quality
    • Method length distribution,
    • Class size distribution,
    • Cyclomatic complexity distribution,
    • Code Smells,
  • Product quality
    • continuous integration successful build trends,
    • test coverage trends for unit tests, integration tests and -ility tests. This metric implies the development team uses unit testing, TDD, ATDD, and automatic -ility testing,
  • Architecture
    • Separation of API and implementation - can only be inferred on modern technology stacks -,
  • Work techniques
    • Refactoring trends - sometimes needs to be evaluated manually in some projects -,
    • Emergent Architecture documentation trends - probably needs to be evaluated manually in most projects.

These metrics shall be automatically provided through your quality tracking and continuous integration systems. For example the "best of breed" tools SonarQube and Jenkins have build-in reports for these metrics, so the tools you use should also have them.

Remainder

The above metrics are tools to discover discrepancies and weaknesses in your product development process and generated artifacts. The inferred results shall be used to identify and implement corrective measures.

Agile emphasize transparency. The above metrics and KPIs shall be visible anytime to all involved participants through dashboards.

Agile GIS Open Data Initiative

2018 03 02 head

How can agile approaches be applied to deliver solutions in a government environment?

Here an example of a GIS open data portal successfully developed using agile approaches such as Kanban, lean product and technical excellence.

GIS Open Data Initiative

Swiss government has initiated an open government data strategy to provide data at federal and state level to all citizens and interested parties. The aims are

Release of official data
The Confederation will make data which is suitable for OGD available to the public in machine-readable and open formats to be freely reused. The individual administrative units must review and, where necessary, adapt the legal, organizational, financial and technical framework of data production to make this possible.
Coordinated publication and provision of official data
With the help of a central infrastructure (OGD portal), the Confederation will make its data sets easily locatable for the public and will make the data available for free use.
Establishing an open-data culture
The Confederation will support the use of data through free, uniform and understandable terms of use and through additional information on the individual data sets. It will also pursue a continuous dialogue with the public.

You can find more details under Open Government Data Switzerland.

The solution realized with agile approaches is a GIS open data portal supporting self-service and providing over hundred theme maps with GIS data in multiple formats, resolutions and geographical sizes available through a self-service portal.

Agile Approach

The solution was implemented during year 2017 and released to public end of year.

We identified these key elements to explain the success of the whole product development. The whole organization and all involved managers shall

  • trust the team will solve adequately and pragmatically the customer needs. Trust your team and create a work environment to support them. For example you should
    • Promote technical excellence,
    • Provide approaches for face 2 face communication,
    • Publish Kanban board to increase transparency,
  • Act as an agile leader
    • Encourage accountability and self-management,
    • Delegate and truly support technical autonomy,
  • Be a gatekeeper to support and protect your team
    • Reduce tension with IT, legal and management systems and build bridges to these departments,
    • Set reasonable and achievable goals.

The delivered solution was

  • Integrated in existing GIS portal of Zürich state to minimize barrier to interested parties,
  • Self-service approach to minimize running costs,
  • Private cloud approach due to regulatory aspects,
  • REST service to provide human readable and machine readable maps in various formats,
  • Build on existing approaches to generate GIS data to minimize risks and respect ambitious timeline.

A description of the technical solution in German can be found here. The REST API is described here.

Learnings

Pragmatic approaches to become agile were very successful in this setup. These governmental organizations were willing and eager to try them.

Senior management was supportive and after initial doubts the team embraced the approach. They enjoyed the whole process and the successful deployment of the solution.

The enablers were * Senior management committed to the approach and supported it, * Team was open to new ways of working, * Team members were competent.

Java 10 aka 18.3 Changes

Java 10 aka 18.3 Changes

2018 03 01 head

For Java to remain competitive it must not just continue to move forward — it must move forward faster.

Mark Reinhold

What has changed?

Oracle has decided to release a new JDK two times a year.

The next release Java 10 aka 18.3 will be available on of March 2018.

JDK 11 aka 18.9 will be released in September 2018. Java 11 will also be a long term support LTS release.

The Visible Change

The visible change is the dawn of the keyword var. Now instead of typing the type of a local variable - local to a method, a block or an instruction - you can just use the keyword var. The constraint being that the variable is initialized at declaration. Stay tune for more support in JDK 11.

When the type declaration is a complex generic type definition the var shortcut is a boon. The code is less verbose and the legibility is at least as good. When the type is a class name I got the impression the benefit is less clear.

As an exercise I updated the source code of open source libraries I have written and the associated tests as in

Event <Events> event = new Event<>(Events.AA_AB);
var event = new Event<>(Events.AA_AB);

The first impression is that the new construct sometimes simplify the code of the library and very often improve the test cases. Perhaps it takes time until I am accustomed to the new way of writing Java code.

Under the Hood Changes

The source code of the JDK is consolidated in one repository. It makes development work more efficient.

The garbage collector shall have a clear and documented API. Developing and adding new garbage collectors is easier.

Various updates such as Unicode newest version. The extensions are used in library classes to better support locales.

Open JDK and Oracle JDK are slowly merging. For example CA certificates will be provided to OpenJDK by Oracle.

Outlook

The rhythm of releases is staggering higher. Starting Spring 2018 we will have * The new rule as blogged by Mark Reinhold Moving Java Forward Faster. * You will get a new JDK every six months * You will get security updates every three months * You will get a long term support release every three years.

This means that you shall update your JDK in development environment six times a year if you want to install the new JDK and the security releases. We recommend automating the process in your environment. And you shall deliver your applications in Docker containers to support the whole ecosystem.

The tool landscape is still struggling to adapt to the new rhythm. I am working with IntelliJ IDE and I had to wait for the EAP 2018.3 before being able to write JDK 10 code in the IDE. I use Gradle as build tool and also to wait until JDK 10 was at least partially supported. And I am still waiting for the update of code coverage plugin. Before that I was back to stone age, writing software with a text editor and compiling it with the command line.

The library landscape is often behind and regularly they are incompatible with the newest version of the JDK.

JavaFX will be removed from the regular JDK distribution in Fall 2018.

The cloud landscape does not support latest JDK as SAAS. You must provide your custom docker images with the expected JDK or JRE. So we are back to IAAS.

Six Levers for better Agile Product Development

Six Levers for better Agile Product Development

2018 02 02 head

How can you create better products and services?

How can you delight your customers, your collaborators, and your stakeholders?

We identified six levers to multiply the effectivity of your agile development teams and empower them to create great services and products.

Please add these tools to your toolbox and create great products.

Design Thinking and Customer Experience

You solve customer problems through intuitive services and products. Customers' expectations should fit with their experience with your products. That is why they should be involved in the whole definition and realization process. At the beginning, to understand the need and to test first ideas. During the implementation, they are directly asked for feedback, which then flows into the further development steps.

Go out with your incomplete product as soon as possible. Each iteration should deliver an upgraded version of the product. The pressure to deliver is huge, a design thinking iteration has a duration of at most one week.

Lean Startup and Lean UX

Minimum viable products - MVP - are first developed as minimally functional products. Within 60 days, a physical prototype will be created that customers can test during this time. The software component shall be delivered after the first or second iteration, that means in less than one month.

New technologies such as 3D printing, cheap hardware and IoT boards, continuous delivery, simulation enable your teams to deliver on time. These are not pure mockups but executable applications. If it appeals to the customer, the MVP will evolve gradually to a minimum marketable product - MMP -. The focus is always on adding customer value.

Beyond Budgeting Approaches

There is no long-term budget plan for products and projects. After each cycle of customer feedback the goals and budget are evaluated again. The value streams have agile teams that work like small companies.

Each team has a Product Owner - PO - who has overall responsibility for their respective product. Realizations are further funded, modified or canceled on the basis of initial results, meaning after customer and technical tests.

The product financing is following a start-up approach and not based on yearly budget as usual in larger companies.

Co-location and Team ownership

Work is done across departments, in cross-functional teams. All those who can contribute to the success of the product (e.g. company organization, IT, market management, or sales) sit in one room. They form the agile team. Co-location ensures lean coordination processes as well as speed and quality.

Co-location also means: You are moving. The teams move into a room and work focused together on their product. There are no phones, no other projects, no outside distractions. They use collaboration tools - chats, wikis, Scrum boards -. Scrum teams are wholly focused on their product,

People checks three times a day their emails; otherwise the email client is closed and inactive. All phones have voicemail.

Agile Methods and Craftsmanship

Employee training is essential. In the value streams, we provide staff with experienced coaches to build knowledge in agile methods and software craftsmanship.

Work is done according to proven agile methods. Software development relies on modern and agile techniques such as pair programming, refactoring, test driven development, continuous integration, and Scrum. For bigger product development we use LeSS approach.

Read this {url-articles}2018/blog-2018-01-01[blog] for a description of modern agile development procedures.

Elastic Infrastructure

We shall use state of the industry technology. This enables test-driven development - ATDD, TDD -, meaning predefined automated software tests that run in real-time.

Cloud infrastructure provides unlimited processing capabilities for development, test, and deployment of software artifacts.

Developers have fast notebooks, fast SSD, plenty of RAM, and 4K external screens. In addition, the technical infrastructure enables software to be scaled and adjustments made immediately - CI, CD, DevOps -. The goal is to deliver a software release in production every few hours if needed.

Learnt Lessons

  1. The power of co-location is tremendous. The work atmosphere is focused, friendly and productive. Team members are continuously working on the solution without spurious task switches in their team room,
  2. Establish stable and long living teams,
  3. Realize the importance of training and coaching,
  4. Embed your teams and processes in the organization,
  5. Change processes for beyond budgeting approaches and lean enterprise. The organization has to stop requiring exact budgets and detailed yearly planning,
  6. Reflect on the impact on the whole organization. See also my blog discussing what agile companies are.

Scrum Guide 2017: What changed?

Scrum Guide 2017: What changed?

2018 02 01 head

The Scrum guide was revised in 2016 and got clarification concerning the core values. I was thrilled. Talking about values and mental models you must have to successfully introduce and be agile is awesome.

A new revision was published end of year 2017, just a year after the values addition.

What has changed? What have the community learnt and committed to the new version of the guide.

Scrum is not only for Software

Scrum was initially developed for managing and developing products. Starting in the early 1990s, Scrum has been used extensively, worldwide, to:

  1. Research and identify viable markets, technologies, and product capabilities,
  2. Develop products and enhancements,
  3. Release products and enhancements, as frequently as many times per day,
  4. Develop and sustain Cloud (online, secure, on-demand) and other operational environments for application use,
  5. Sustain and renew products.

The essence of Scrum is a small team of people. The individual team is highly flexible and adaptive. These strengths continue operating in single, several, and networks of teams. They develop, release, operate and sustain the work and work products of thousands of people.

They collaborate and interoperate through sophisticated development architectures and target release environments.

Scrum is a tool to solve complex problems, any kind of complex problem. It is already the industry standard for software projects, it is becoming a major standard for all complex products.

What does a Scrum Master?

Changed wording in The Scrum Master section to provide better clarity to the role. The text now reads:

The Scrum Master is responsible for promoting and supporting Scrum as defined in the Scrum Guide. Scrum Masters do this by helping everyone understand Scrum theory, practices, rules, and values.

The Scrum Master is a servant-leader for the Scrum Team. The Scrum Master helps those outside the Scrum Team understand which of their interactions with the Scrum Team are helpful and which are not.

The Scrum Master helps everyone change these interactions to maximize the value created by the Scrum Team.

Added to the section Scrum Master Service to the Product Owner

Ensuring that goals, scope, and product domain are understood by everyone on the Scrum Team as well as possible.

Scrum master is a servant leader and promotes Scrum and agile approaches. The whole team work on a product. They understand the vision, goals, scope, and have extensive domain knowledge.

This means the whole team talks with stakeholders and users. The team members have a genuine interest to learn more about the product domain.

Scrum Events

Updated the first paragraph of the Daily Scrum section to read
The Daily Scrum is a 15-minute time-boxed event for the Development Team. The Daily Scrum is held every day of the Sprint. At it, the Development Team plans work for the next 24 hours. This optimizes team collaboration and performance by inspecting the work since the last Daily Scrum and forecasting upcoming Sprint work. The Daily Scrum is held at the same time and place each day to reduce complexity.
Updated the Daily Scrum section to provide clarity on the goals of the Daily Scrum including this text
The structure of the meeting is set by the Development Team and can be conducted in different ways if it focuses on progress toward the Sprint Goal. Some Development Teams will use questions, some will be more discussion based. Here is an example of what might be used:
  • What did I do yesterday that helped the Development Team meet the Sprint Goal?
  • What will I do today to help the Development Team meet the Sprint Goal?
  • Do I see any impediment that prevents me or the Development Team from meeting the Sprint Goal?
Added clarity around time-boxes
Using the words “at most” to remove any questions that the time-box for Events means maximum length, but could be shorter.

The guide clarifies the daily Scrum event. It establishes a daily feedback loop for improvement.

For slow learners time-box means maximum duration.

Scrum Artifacts

Added to the Sprint Backlog section
To ensure continuous improvement, it includes at least one high priority way in which the team works, identified in the previous Retrospective meeting.
Added clarity to the Increment section
An increment is a body of inspectable, "Done" work that supports empiricism at the end of the Sprint. The increment is a step toward a vision or goal.

The guide emphasizes the learning and improvements process. One root of the agile manifesto is lean thinking.

For slow learners you inspect the result of the increment and put at least one improvement measure in the next sprint backlog.

Previous Changes

The biggest change for the daily work was introduced in the previous revision - read previous blog -. The word commitment is no more used in the description of the planning event. The team now provides a forecast which stories shall be realized until the end of the sprint.

This change was necessary because people - especially command and control responsible - did not read the official definition of commitment - Oxford Dictionary -.

The state or quality of being dedicated to a cause, an activity.

The second big innovation was the introduction of five Scrum values: Commitment, Focus, Openness, Respect, and Courage.

I welcome the focus on values, principles and core behaviors over detailed checklists and rules. Please also read again the twelve principles of the agile manifesto.

One Way to Improve Your Scrum Approach

One Way to Improve Your Scrum Approach

2018 01 02 head

You work using agile approaches. Your collaborators are trained how to use agile approaches. Techniques such as test driven development, continuous integration or clean code are regularly practiced.

Now you are looking for the next improvement. How can your team improve product and increase customers' value?

It is time for a fresh view. Perform an agile assessment with an external expert from another company or department.

Effort

You want to take a fresh view on your process and practices. The external person shall provide hints that he observed in your team and how other groups solved similar problems. The assessment lasts approximately 2 days per team including the finding workshop with the team members, Scrum Master, and Product Owner. This workshop has around half a day duration.

Approach

The Scrum Assessment speeds up your Scrum mastery and let you collect low-hanging improvements.

  • Do you already work with Scrum or Scrum/Kanban and want to know where you stand?
  • Do you get the benefits that are possible with Scrum?
  • Where do you deviate from the Scrum theory and what are the consequences for you?
  • Are your structures and practices for coordinating dependent teams effective, meaning scale agile?
  • What potential for improvement do you have?
  • Which measures make sense so you can benefit more from Scrum?
  • Could you introduce agile techniques to increase the quality and to enhance outcomes?

We work on these themes in the Scrum Assessment.

  • We analyze your Scrum implementation,
  • We collect feedback through questionnaires and observation,
  • We assess a single team or the whole development organization,
  • We evaluate in comparison with other companies and make suggestions for improvement.

Action

We typically proceed as follow for the analysis:

  • Record Scrum meetings - sprint planning, sprint review, sprint retrospective, daily scrum, if necessary backlog refinement -,
  • Observe how the team members work during the sprint, to better understand the dynamics in your team,
  • Analyse the vision, roadmap, release plan, product backlog, and impediment list together with the product owner,
  • Interview product owners, scrum masters, selected developers and selected stakeholders,
  • Review the structures, processes and practices (e.g., Scrum of Scrums, Portfolio Kanban) used to coordinate dependent teams.

After this analysis, we do a debriefing in a half-day workshop. We present our observations, conclusions and recommendations. The Scrum team uses our findings to reinitiate their agile transformation journey.

Through the workshop approach, we tailor the recommendations together with the team members to specific needs and we can clarify any ambiguities.

Upon request, we will also prepare a written report.

Why is it so Cool to Develop with Java?

Why is it so Cool to Develop with Java?

2018 01 01 head

Why is it so cool to develop software using Java, a modern IDE and agile tools. Because the environment empowers you and incredibly speed-up your development outcomes. In seconds, you know if the next code increment adds value to your product.

Liberated your team works adding functions and no more spend unaccounted hours in setting up infrastructure.

Below our current set of techniques and tools to multiply our results and bring earlier a product to your customers.

Java Quality

Solution Quality

The most efficient approaches to improve the intrinsic quality of your software solution are

Use Pair Programming
Write regularly together source code. Decide that all complex code parts shall always be developed through pair programming. If you feel adventurous give a try to mob programming,
Use Refactoring
Before pushing your code to the Git repository, always take a look and refactor your code. You are a craftsman and shall be proud of your result,
Use Java 8 Lambdas and Streams
Actual Java code builds on lambda expressions and stream constructs. Your for loops are replaced by stream pipeline, your algorithms extensively use lambdas.
Use Java 9 Modules
It is finally possible to clearly define public interfaces of a whole software component. Configure the dependencies to other libraries, and identify specific modules as privileged participants.

Do not write Java code as it was done last millennium. Use the actual constructs to write solid and resilient code.

Code Quality

The most efficient tools to detect glitches or legacy constructs in your code are

SonarLint
The plugin for your favorite IDE provides all the SonarCube checkers to apply on your source code. Purify it before pushing it into the continuous build pipeline. The plugin support download of tailored rules from you company SonarCube instance,
SpotBugs
The successor of the wildly popular FindBugs tool can be integrated in your build pipeline to improve the odds of correctness,
PMD
Is another tool to automatically detect potential quality problems,
IntelliJ IDEA build-in Checks
You are lazy and do not want to use additional tools, please use at least the build-in checkers of IntelliJ IDEA to improve your code. It is very unprofessional to have equality problems or boolean condition errors in modern code.

Test Quality

Test Driven Development - TDD

You are serious about development quality code. So you are working with the test driven approach TDD and need to write unit tests. Each time you write some code you go through the loop "red - green - refactor - commit".

Junit 5
is the standard and best unit test framework for Java solutions. The newest version support lambdas is a pleasure to use.
AssertJ
enables the writing of legible assert statements in your unit tests.
Mockito
Tests must be lightening fast, thousands of unit tests shall run in a few seconds. Remove all dependencies to externals components from your source code, mock these elements with Mockito.
Hamcrest
Your test conditions are complexer and you want to have legible code, start using hamcrest to write clear assertions in your tests.

Please never access a database or a file system when running unit tests. If you low level tests with files just use an in-memory file system mock. [1]

Each time an error is found in productive code, write first a test to reproduce it, then correct the error. This approach guarantees the same error will never again occur.

Acceptance Test Driven Development - ATDD

You are serious about delivering the functions your customer need. Each story you implement has acceptance criteria. These acceptance criteria defines your requirements - see specification by example -. Your acceptance tests document and validate these requirements.

Cucumber
Write your acceptance criteria and document your requirements in one step.

Remember stories are not requirements, they are work items. Requirements are coded and documented in your acceptance tests.

Continuous Integration

Agile means reacting to changes. Lean means build-in quality. Your continuous pipeline certifies after each change that your solution is correct and has the correct behavior.

Gradle
Is the new standard for building your artifacts.
Jenkins
Is the regular tool for local build pipeline.

If you a choice use a cloud continuous integration for team activities. Decide if you prefer the cloud approach - e.g. with TravisCI or Bitbucket Pipelines - or Jenkins within Docker for your local integration activities.

A smooth running CI pipeline is a boost for team morale and motivation. To know your application compiles and runs through all your automated tests anytime is such a good feeling.

Continuous Delivery and Deployment

Agile means react fast the changes. You need to deliver and deploy your new solution to your customers in minutes or hours.

Docker
Virtual machines were a powerful construct but they are slow and require a lot of resources. Docker is the answer. Start a testing instance in 300 milliseconds.
Tomcat and TomEE
Embedded servers deployed in docker image in a matter of seconds. Actual versions of Tomcat or TomEE support it.
hsqldb
In memory database are lighting fast for integration and test staging areas. Later move to a file based database for the production area. The database shall be a configuration of the delivery and not require a new compilation.
Google Cloud AppEngine
To deploy various integration environments

Design Quality

Important decisions and overall structure shall be documented for new team members. Architecture Decision Record: A lightweight approach to document architecture decisions using markdown documents and traceability through the decision history. UML: Sometimes I would like to draw some UML diagrams to explain a design solution. I currently using a textual tool to define such diagrams.

Development Quality

Analog Board
Nothing beats a big analog board in the team room as the ideal information radiator. If you want your colocated to work effectively and solve problems as team, you shall provide an analog board.
Trello, Stride, BitBucket, Hipchat and others Atlassian tools for distributed teams
The first advice is to have a collocated team. The productivity and performance gains are unachievable in a distributed environment. We have very good experience with Atlassian and Google App solutions to mitigate the drawback of distributed teams and establish a team work atmosphere. But a colocated team still rocks.
IntelliJ IDEA
This IDE is the choice for refactoring, powerful context suggestions and efficient programming. If you use Eclipse or Netbeans just try a friendly refactoring competition with an IDEA convert.
Git
Please stop using Subversion, CVS, TFS or other geriatric tools. Agile development, efficient integration in delivery pipelines, and extensive refactoring require Git. Just migrate now to Git and leave dreaded message Tree conflict detected, please solve manually behind you.

These tools shall be used the agile way. You do not have long-lived branches in your source code management tool. Either you have only short-lived local branches or better you just work against trunk. Complex branching strategies are just waste for a lean practitioner.

I decided to collect all these hints in the same long post. They belong together. Every day you design a solution, you code it, test it, improve it and guarantee its quality.

These activities are tangled together to produce a product you are proud of.

And yes I know about Github, Gitlab, Slack, Ant, Maven, Jetty, Amazon Cloud, AssertJ, Subversion, Eclipse, Netbeans and other good tools. But I still prefer the ones above; I am convinced to be more productive with them.


  1. After years of neglect, a Jimfs 1.2 was released in January 2021 and published on maven central.