As of October 1, 2023, LINE has been rebranded as LY Corporation. Visit the new blog of LY Corporation here: LY Corporation Tech Blog

Blog


Agile2018 Recap

Hi there! My name is Hiroyuki Ito (The HIRO), a SET (Software Engineer in Test) and Agile Coach from LINE. This summer, I attended Agile2018 held in San Diego, a shiny beautiful city in California, U.S.A!

Gate.jpg

What Is Agile2018?

Agile Alliance annually holds the biggest global conference on Agile, in the U.S. Here are numbers of Agile2018; you can tell how big this conference was.

Date August 6th–10th Five days
Number of Sessions 279 Five more compared to last year
Number of Submissions (Proposal for sessions) 1,500 Experience Reports: over 100 submissions
Number of Categories 21 The same as last year
Number of Parallel On-going Sessions 20 One more compared to last year
Number of Attendees 2,400 From 54 countries
Type of Sessions
  • Talk
  • Workshop
Level of Sessions
  • Learning
  • Practicing
  • Advancing

And here is the list of categories and sessions of this year's conference. We had to choose sessions to attend from the list. It was very difficult!

Category No. of sessions Category No. of sessions
Agile Companies 16 Keynotes 3
Agile Data Metrics and Forecasting 6 Leadership 21
Agile Foundations 7 Learning 19
Agile Midway 7 Project Program & Portfolio Management 18
Coaching & Mentoring 21 Testing & Quality 12
Collaboration Culture & Teams 24 User Experience 12
Customers & Products 15 Experience Reports 24
Dev Practices and Craft 17 The Future of Agile Software Development (IEEE Software) 8
DevOps 12 Audacious Salon 14
Enterprise Agile 20 Stalwarts 10

The venue was Marriott Marquis San Diego Marina. We enjoyed beautiful views. I stayed at this hotel and I liked the design of the room key.

Hotel.jpgViewFromTheRoom.jpgRoomKey.jpg

Sessions

Although I have already published a flash report of the conference, I would like to share the details of each session I attended at the conference with you.

Day 1

Day 2

Day 3

Day 4

Day 1

Of the sessions I attended on the first day of the conference, I'd like to share two of them with you; Learning to Experiment and Show Me the Money! Defining Enterprise Value at Scale.

Learning to Experiment

This talk was presented by Chris Lucian(@ChristophLucian), the director of software development from Hunter Industries, and the founder of Mob Programming. He shared his experience in a lot of experiments conducted at Hunter Industries.

His main agenda was refining the process continuously without emergency. He said that he experienced the best organizational changes because of emergencies in his career. However, emergencies are not ordinary. He has been supporting to improve the process of experimentation in production work without emergency.

WithChrisLucian_1.jpg

He created the foundation of psychological safety for experiments. His team takes just seven hours per week for learning sessions, run retrospective meetings very frequently and obtain budget for anything that can contribute to software development. In addition, he visualized the progress of each experiment for business personnels for their safety during the experiments.

Here are some experiments that have been conducted at Hunter Industries:

CD + CI (Continuous delivery + Continuous integration)

  • In the inception phase of the Mob Programming, the team in Mob programming, focused on continuous delivery of values over estimation.
    • This became the first catalyst of improvement.
    • Support is provided by the Product Owner and Ops team.
  • During the growth phase of the Mob, they improved CI to support the growth.
    • Database CD with Flyway
    • Automated acceptance tests to earn domain knowledge
  • Cycle time is a good competitive advantage: Release small slices of features quickly.

Kanban for Management: Focus on management tasks

  • Kanban for Management worked.
  • The speed of work slowed down due to the growth and the expansion of the company.
  • They are now experimenting "Home Owners Associations" (optional management-related groups).

Lofty Goals: Literally lofty goal with Working Agreement

  • Defined lofty goals loosely at first.
  • Diluted due to the growth and the expansion of the company.
  • Improving and redefining it continuously now. Expanded the target from a team to a department.

Some succeeded, some failed. However, they didn't stop because they failed. They improved some of the experiments after failures. They went forward through failures.

I've learned the following in his session:

  • It takes long to improve processes and organizations.
    • It took 6–7 years at Hunter Industries.
    • We Agile practitioners better be patient.
  • Not Mob Programming only: Chris is famous for Mob Programming. But he is utilizing a lot of other practices.
  • Actions for outcome
    • Delivering value continuously became the first catalyst of improvement at Hunter Industries.
    • Their experiments and improvements are always focusing on outcomes.

Here is more information of Chris's story:

Show Me the Money! Defining Enterprise Value at Scale

This workshop was led by Gail Ferreira(@LeanAgilist), the Principal from Boston Consulting Group. In this session, we learned how to show business values to stakeholders in order to obtain funding. By the way, in this session, the word "scale" meant "scaling Agile to the business persons" .

ShowMeTheMoney_1.jpg

We were given the following tasks at the workshop:

  1. Create a group with 5–6 persons.
  2. Decide the target business.
  3. Identify and define values which can attract fundraisers.
  4. Define MVP (Minimum Viable Product) to achieve the values identified and defined in step 3.
  5. Define the points of innovation and improve our MVP.

Here are our MVPs. Our business was a special catering service for global conferences and events (which Agile practitioners long for!)

ShowMeTheMoney_2.jpg

In this workshop I learnt that considering and showing business values to business personnels in and out of the company is necessary throughout the product lifecycle. Also that we can innovate our products by getting ideas from other team's products.

Here are a couple of links for you to take a look at:

Extra session with Chris Lucian

At the party on the first night, I and other Japanese attendees got a chance to meet Chris Lucian (@ChristophLucian) in person. And... we were lucky to have an extra lesson from him!

WithChrisLucian_2.jpgWithChrisLucian_3.jpg

Here are the things we discussed about Mob Programming:

  • Now there are 8 mobs collocated at Hunter Industries. Mob emerges for each project/product with a flat structure.
  • Mob can expand communication bandwidth. They don't do estimation and daily meeting.
  • We should not change the mob and its members simply to keep communication cost low.
  • Mob can build autonomy by:
    • Doing small
    • Getting feedback
    • Iterating quickly
  • Key-binding is NOT a barrier for Mob Programming. I once tried Mob Programming and some programmers opposed it because key-bindings differed on each PC. Mob can embrace it by learning other person's key-binding and/or integrating key-binding within the mob.

He has cautioned us not to be practice envy, to avoid complacency not to wait to experiment. We've also shared a bit about philosophy; examples and real solutions are the things we should value more, not logics and books, status quo means the start to lose and also that we can use transparency(example: metrics) as a tool for collaborating with each other.

He's shared that they are using Kanban for all projects and products at Hunter Industries, and they don't scrum. It seems that it is very hard too to adapt to Agile in the U.S. Many people in the U.S. tend to persist in their role, create silos, and blame for others, due to clear job descriptions and responsibilities of positions. This issue exists in the Japan, too.

Day 2

On the second day of the conference, I attended two workshops; Deal Me In! Portfolio Dashboard Poker – A Fun Way to Align Business & Technical Goals and Database DevOps: Strategies to Address DevOps' "Last Mile".

Deal Me In! Portfolio Dashboard Poker – A Fun Way to Align Business & Technical Goals

This was a workshop presented by Jason Tice(@theagilefactor), the Vice President of Business Innovation from World Wide Technology. In this session, we learned how to discuss Portfolio Management with Portfolio Dashboard Poker.

DealMeIn_1.jpg

The tasks we were given were:

  1. Create a group with 5–6 persons.
  2. Choose a scenario from 16 scenarios.
  3. Choose proper metrics for the chosen scenario from 100 cards, consisting of business metrics and technical metrics. Choose up to 10 metrics consisting of at least four business metrics and four technical metrics.
  4. Prioritize metrics we chose.
  5. Clarify why we chose those metrics.
  6. Classify metrics between trend and target.
    • Trend: Metric as signal to check the trend
    • Target: Metric as the target of the product
DealMeIn_2.jpgDealMeIn_3.jpgDealMeIn_4.jpg

Here are our metrics. I've learned that predefined simple metrics are very useful as a starting point of communication. Before we start and facilitate communication, predefining ideas that are often used, and using these in communication would help. This workshop was good for establishing shared understanding between business personnels and technical personnels. I could see that ideas tend to differ depending on the role of the person, but I believe this contributes to diversity.

DealMeIn_5.jpg
Business ones
DealMeIn_6.jpg
Technical ones

Database DevOps: Strategies to Address DevOps' "Last Mile"

This workshop was held by Scott Ambler(@scottwambler), the Senior Consulting Partner from Scott Ambler + Associates. He is famous for Agile Modeling, Agile Data, and Disciplined Agile. In the workshop, we considered and discussed the way to make it easy to change production database. I think that persistence layer is the key concern for DevOps. I was very keen to attend this session.

DatabaseDevOps_01.jpg

The tasks were:

  1. Create a group with 5–6 people.
  2. Discuss about challenges we are facing for database and/or DevOps and share the solutions. Write them down on sticky notes.
  3. Pick up three challenges and three solutions and post them on the whiteboard.
DatabaseDevOps_02.jpgDatabaseDevOps_03.jpg

After completing the given task, Scott shared strategies for database DevOps:

  • Vertical Slicing
  • Clean Architecture/Design
  • Agile Data Modeling
  • DB Refactoring
  • DB Regression Testing
  • Continuous Database Integration
  • Config Management
  • Operational DB Monitoring

DatabaseDevOps_06.jpg

And... we found a paper on our table, which explains one of the strategies he shared. The strategy we were given was, "Database Refactoring".

DatabaseDevOps_07.jpg

Now another set of tasks were given:

  1. Discuss the strategy on the paper from the following aspects, and write them down on sticky notes.
    • Advantages
    • Disadvantages
    • The way to gain skills for the strategy
    • Tools which support the strategy
  2. Pick up three sticky notes in each aspect and post them on the whiteboard as follows.

DatabaseDevOps_10.jpg

And here are the ideas gathered for other strategies.

DatabaseDevOps_08.jpgDatabaseDevOps_09.jpgDatabaseDevOps_12.jpgDatabaseDevOps_13.jpgDatabaseDevOps_11.jpg

Scott emphasized that changing schemas of production database is NOT difficult with:

  • Simple design
  • Evolutionary database development
  • Lightweight practices
  • TDD(Test-Driven Development), Refactoring, Regression Testing for database

The way our workshop ran proved how the wisdom of the crowd encourages shared understanding. I would like to apply this style at LINE later. Ah, if you are interested in refactoring database, here is a book for a read, Refactoring Databases: Evolutionary Database Design.

Day 3

What is the story with Agile data?

This talk was given by Troy Magennis(@t_magennis), the founder of Focused Objective. This was an opportunity to learn the latest knowledge on metric, in other words, Agile data. The key idea of this session was that data shows stories behind it. This idea is very important for us to choose right actions based on data. Watching this video, The Joy of Stats, will help you understand this idea.

Keynote_1.jpg

I have been making use of metrics since 2013. I was able to validate that my ideas(Paper at AsianPLoP 2017, Japanese) and activities of metrics are almost the same as this session. Once again, I was able to confirm that data is useful also for experiments.

As to the question of whether we should choose story points or throughputs (item counts), Mr. Magennis said that it depends. Story points work for development time rather than delays, and throughputs work for delays rather than development time. He recommended us to gather the recent ten samples. He pointed out that it is NOT the absolute ones that are useful to us; we always need to check which data is useful to us.

Here are some aspects of data he shared with us:

People Problem

People need data because learning only from experience leads to bias. The reasons of gathering data are to get status, for planning and cherry-picking for senior members. We need to gather useful data, like history. It is the best to avoid "Watermelon Status" (= Full of red (= warnings) coated by green). We need safer data to make people happier.

Keynote_2.jpg

We need to change the priority we cast on our analytical effort as below. In other words, we need to focus more on outcome, rather than output.

As-is To be
  1. Status/Progress
  2. Selection/Prioritization
  3. Customer Validation
  1. Customer Validation (customer value)
  2. Selection/Prioritization
  3. Status/Progress

Predictability

There is always uncertainty. We need to handle uncertainty with data. With data, we can reach a goal step by step. We can find ways to move forward by data. Let's have a look at an example, Google Map. Google Map provides multiple options for context. It provides duration, not ETA (estimated time of arrival). It continues to update information until, say, we start finding a way. This is totally different from software planning methodologies of these days; we only have a single option, our planning is based on calendar date, and we stay doubtful until the end of it.

Forecasting

Forecasting is about knowing when to start. The number one reason why we miss deadlines is we start too late! Forecasting is about detecting we are wrong, not for we are right.

Day 4

On day four, I attended a workshop, Stepping Stones in Refactoring to Microservices , and a talk, Creating Chaos … Engineering.

Stepping Stones in Refactoring to Microservices

This workshop was led by Amr Noaman(@AmrNoaman), the author of Refactoring to Clean Code. He presented three stages of refactoring microservices; Quick Wins, Divide and Conquer, Inject Quality in.

Quick Wins

  1. Remove dead code
  2. Remove code duplication
  3. Reduce method size
  4. Enhance identifier naming

Divide and Conquer

  1. Split the code into modules and reduce coupling
  2. Discover and grow physical and replaceable components
  3. Promote components to services or microservices
  4. Useful grouping themes:
    • Business or Domain
    • Utility
    • Port
  5. Model
    • Strategy of storage for microservices:
    • Divide storage per microservice
    • Synchronize (integrate) storage among microservices

Inject Quality In

  • Cover components with automated tests
  • Enhance components internal design

RefactorMicroservices_1.jpg

After learning the basics, we discussed benefits and risks of the following, wrote them down on sticky notes, and posted them on the whiteboard. We used the wisdom of crowds again.

  • Microservices
  • Service-based architecture
  • Component-based architecture
RefactorMicroservices_4.jpgRefactorMicroservices_3.jpgRefactorMicroservices_2.jpg

If you are interested in microservices, check out this book, Building Microservices: Designing Fine-Grained Systems.

Creating Chaos … Engineering

Shahzad Zafar(@m_shahzad_z), the Vice President of Engineering in Rx Savings Solutions presented this talk. He talked about chaos engineering. To tell you the conclusion first, I learned that Domain-Driven Design (DDD) is required to understand both Chaos Engineering and microservices.

ChaosEngineering_1.jpg

Before starting the session, he asked us to fill out a questionnaire about the images we have of chaos engineering. And here is the result.

ChaosEngineering_2.jpg

He introduced the purpose of chaos engineering. It is to detect defects in distributed systems; for resilience. A person cannot manage everything. He then went over the history of chaos engineering, and introduced Chaos Monkey, a resiliency tool and Simian Army. In chaos engineering, you move from hypothesis to experiment and validate with business metrics.

Getting started

He shared the basic steps to get started on chaos engineering:

  1. Start with known weakest link
  2. Monitor: First monitor manually and automate later
  3. Expand actions company-wide

Where to start?

He also mentioned that to proceed with chaos engineering, we need to change the following processes first:

  • Organizational risk tolerance
  • Understanding the process of creating hypothesis.

Steps of real experiments

Mr. Zafar recommended to conduct experiments on a real (production) environment. He suggested the following steps for conducting back-end experiements on a real environment:

  • Test failure of a LB or Service
  • Fault testing for an AZ or Region
  • Test failure of an entire rack
  • Power loss vs server shutdown

Scaling beyond a team

He also pointed out that scaling beyond a team is necessary and shared the steps for it as follows:

  1. Move from “the shadows” to invested
    • Pilot: Small & Stealth
    • Getting broader buy-in
  2. Create an automation tool with the following features:
    • Canary analysis
    • Monitoring
  3. Get to a point where running an experiment needs to be routine and not time consuming

Conclusion

He concluded with the following tips:

  • Start small, grow from there.
  • Spend time writing your hypothesis.
  • Automate and build-in needed capabilities.
  • Recognize risk tolerance.
  • Running experiments all the time.

[Bonus] Photos with Gurus

With Woody Zuill, the founder of Mob Programming. We talked a lot during this conference. Additionally, he gave me a lot of advice, insights. It was fun talking to him!

WithWoody_1.jpg

With Linda Rising, the author of Fearless Change. I always keep her advice in mind, "showing values continuously is the best way to improve".

WithRinda.jpg

With Stas Zvinyatskovsky, the keynote speaker of DevOpsDays Tokyo 2018, and David Bernstein, the author of Beyond Legacy Code.

We talked about:

  • Test scripts for sharing specification and support communication among team members and stakeholders.
  • DevOps as a way of Psychological Safety.
  • Automation techniques for contributing to business outcomes.
WithStas.jpgWIthDavid.jpg

With Per Molsing Beining, the Agile Coach from Denmark, and Martin Heider, the Agile Coach from Germany. We agreed that both technology and process are necessary for Agile Transformation.

WithPer.jpg

With Jutta Eckstein, my shepherd at Agile2014. She recently wrote Company-wide Agility with Beyond Budgeting, Open Space & Sociocracy: Survive & Thrive on Disruption.

WithJutta.jpg

Closing

LINE employees have a chance to apply to attend global conferences like Agile2018 annually. This is one of million other reasons why working at LINE is great.

Party_1.jpg

By the way, we are looking for SET. You can choose to work on the server-side or client-side, and also choose to work in Tokyo or Fukuoka or Kyoto. If you are interested to work with us at LINE, please apply here.

Thank you for reading!