All work and no play makes jack a dull boy (software engineer)
Pragmatic approach to experimentation through culture, time and environment
Why do athletes 🏋🏻 hit the gym often? - To flex their muscles and stay on top of their profession. Same as athletes, software engineers have to flex their brain by learning through practice and experimentation of their core tech skills. Let it be learning a new programming language, rolling new tech tricks up their sleeves, using a new architecture pattern, trying out a new service in AWS, adopting a new framework etc. developers need to experiment in their area of expertise.
Without play and experimentation, they can’t:
Do critical thinking and suggest an alternative to resolve a high priority problem at hand that they would have learnt and practiced through play and experimentation otherwise
Consult with non-technical stakeholders in building new product features by adopting technologies the team has never worked with before.
Assess the limits of their system as it doesn’t show consistent behavior during chaos or high load
Upskill themselves and feel that they are after all an engineer at the end of the day
Your role as a manager is indeed quite important for your team to embrace learning, experimentation and continuous improvement as cultural elements and provide enough space for them to flex their brain, not just building feature after feature in the same old way. As managers, you were trained and told to deliver value after value at a rapid pace, which is not something I’m against at, however your engineers deserve more than that - having dedicated time and environment to play and experiment with tech skills that they would like to master and eventually that will payback for your team and organization.
In this post, I would like to share my experience and challenges on how I created
Culture 🙏
Time 🕤 and
Environment 🌱 (along with practice)
for my team to play and experiment with technologies in the midst of tight deadlines and motivation behind it.
Don’t miss out on the last part of this post on how to advocate for it to your management team if you were questioned on why your team needed to pursue experimentation since you’re not a laboratory after all.
In “All work and no play makes Jack a dull boy” phrase, I compare “Work” to clearly defined/scoped work (requirements) that contributes to the product and service that your team is building. “Play” refers to experimentation with technologies and unlike clear cut requirements, engineers in your team themselves define what they’re playing with, what they’re trying to achieve and how they want to share the knowledge among other engineers.
This post is not about product experimentation done through MVP, hypothesis and prototypes but purely an experimentation on engineering stuff and technologies that your team is working with.
Culture 🙏
As like anything else, start with the cultural aspect. Without understanding the motivation and how it would impact your people, you can’t bring in a change. You have to look for symptoms when engineers express they weren’t learning, not getting time to practice and to learn anything new etc.
In my teams, there were cases where people felt they were working too much feature after feature and project after project. They felt their tech skills had become rusty and not much of new things they were learning or experimenting as the work has become quite monotonous. As a manager, it’s on you to ensure your team continues to find ways to learn and in fact it’s not difficult at all. Just lend them your support and look for ways to learn in everything your team does. Let it be a success or a failure. Learning should be the ultimate goal and your culture values should reflect that.
To bring in experimentation, understand people’s motivation behind it, why they want to experiment, what they want to experiment and ultimately what they want to achieve.
Safe to fail
First things first. Be ready to fail and embrace failure when your team sets out with experimentation. This is what guarantees your engineers to try out without a fear of failure and encourage them to share learnings and even continue with experiments to achieve what they were aiming for.
If the experimentation is successful, you can turn that into a usage. If the experimentation is a failure, there will be tons of learning that your team will carry forward with and get ready for the next try and down the road your team might crack the code and can be turned into a usage. It’s a WIN-WIN situation. Make sure continuous learning and improvement is always there in whatever you do.
Knowledge sharing
Sharing knowledge through experimentation is valuable for your team and as a manager you should encourage your teammates to share findings and learnings from the experimentation whether it’s a success or a failure. We use guild meetings to share about the experimentation and what an engineer has played with and share that with every other engineer.
Make continuous learning and experimentation as one of your cultural elements and provide necessary support to your team to experiment on technologies that they wish to do.
Time 🕤
Without allocating time, nothing would be possible. Having play and experimentation as part of your team culture is a great start but ultimately you need to help them find some time in the middle of delivering projects after projects.
As an engineering manager, be transparent with your team about how much time is allocated for experimentation and intention behind it. Experimentation doesn’t need to be run at a large scale and for months. It can be simple and focused that your software engineers can yield quick wins and learnings to incorporate it in their day-to-day work as well.
Short bursts 💨
Are you waiting for a dedicated time at the end of the year to experiment and learn? - that time never comes from my experience. Capitalize on regular breaks that you might get in the middle of working on projects. We deliberately work on medium sized technical experiments (to a max of 3 weeks) after every few big sized projects to keep the momentum and get some time for learning.
Dedicated time
80-20% time allocation every sprint could also work where 80% of your team’s time can be spent working on defined projects and 20% on technical improvements, experimentation etc. Compile a backlog of experimentation that your engineers wanted to do and display it in a common place like JIRA or any collaboration tool where everyone can see what’s cooking.
When it comes to experiments, don’t be too nosy as a leader and let your engineers understand to make the time well spent and leave it at that point.
It’s their time, their experimentation and their learning - NO HOLDS BARRED until it doesn’t cause too much stress, money and bad impact on end users.
Hack, Hack, Hackathons 🐱
Hackathon is another great way to facilitate experimentation that your engineers wanted to carry out - but it doesn’t have to stop there. Advocate for experiments and ideas that are worth integrating into your product and services with your stakeholders and other leaders and that gives a great sense of accomplishment and motivation for your engineers.
Environment (Practice) 🌱
Experimentation in Production 😱
Your heart might skip a beat with what I’m going to say next - some teams run experiments in production. Yes, in PRODUCTION. How is this possible? You need the right environments and practices to be set up and followed to enable such a degree of experimentation. In fact, running in production only can ensure simulating the real traffic and thus experimentation can be carried out reliably.
While carrying out experimentation in production, your system and infrastructure should be safe to fail in a way how it’s set up and architected. This will ensure safe experimentation being carried out without impacting your business and end users. Feature flags, A/B testing are ways to run experiments in production and could be handy options to check it out if you’re not using it yet.
Some teams and companies rigorously run experiments to support their large scale systems and to know how it behaves under stress and even take it to extremes by automating experimentation. That’s how one can innovate continuously in the midst of growing scale. Think what level of experimentation is necessary for your systems and plan accordingly.
Tech Radar 📡
Techradar is another great framework to set up experimentation in your teams. It has 4 stages (Trial, Assess, Adopt and Hold) for operating and bringing in new tools, practices and software into your ecosystem. Having this framework in place will set clear expectations, provide transparency and clarity about the state of the technology in your ecosystem and encourage your engineers to propose any experiments that they want to carry out as a “trial” to try it out and “adopt” based on the outcome.
We used the techradar model in experimenting with a few technologies and practices like GraphQL, Timeseries DB, Event storming etc. and brought some of them into practice and put remaining on hold and retired it.
For chaos 😵💫 and scale
Software that you build is uncertain when it comes to stability as they aren’t immortal and in fact susceptible to chaos and other technical challenges anytime. Chaos engineering is experimenting on a system in order to build confidence in the system’s capability to withstand turbulent conditions in production.
Experimentation helps in measuring the scale of your systems and provision resources accordingly during fluctuating load by scaling up/down, communicating benchmarks from experimentation to your engineers and stakeholders to decide whether to re-architect your system or work with limitations in place. Sometimes we do few spikes and experiments as a follow up of post mortems for technical incidents to understand the system better and I can only say that it brought us interesting learnings and insights which we incorporated down the road.
You may run experimentation formally where you set up clear expectations and the end state of some experimentations and on the other hand, sometimes your engineers can be set free to experiment on their area of expertise, where they wanted to experiment, learn and grow in their profession. This will greatly increase their mettle as software engineers and they would appreciate it.
Cost 💰
In terms of cost for experimentation, in my experience it ranges from minimal to negligible cost as we use the same resources or of less configuration just for experimentation and later clean up once the experimentation is done. As like anything, it needs patience to reap the fruits of experimentation and it takes time.
What matters the most at the beginning is willingness to let your engineers be involved in software experiments. They will come up with interesting ideas down the line for the trust and understanding you have shown for their hunger for learning.
Dear Executives 💼
If you’re an executive reading this post, you might question how much value play and experimentation would bring to your organization.
Don’t be too dogmatic when it comes to your engineers trying out new stuff and experimenting on their area of expertise. You may not see immediate business value out of these experimentations but this is what software engineering is all about as it is ultimately a craft. Provide some leeway to your engineers by giving them quality time and environment to play and experiment. Who knows, next day you might be staring at a new business prospect purely driven by technology.
Work on your culture, Give your team necessary time and environment, Let them play and experiment.
Questions to ask❓
I’ll leave you with few questions to ask yourself and your team as an engineering manager:
What was the last technical thing at work that your team has experimented with?
Why should your team be involved in play and experimentation?
What will be your team’s mindset when it comes to experimentation? Are they feeling excited or they feel such a thing never exists?
How do they feel when they fail in their experimentation? and what’s following up after that?
What’s your experience in enabling your software engineers to play and experiment with their area of expertise? What are the challenges and wins that you experienced adopting an experimentation mindset in your teams? - I would love to know your thoughts.
> Who knows, next day you might be staring at a new business prospect purely driven by technology.
You can give real-life examples here, as many well-known products came out of "side project time" https://en.wikipedia.org/wiki/Side_project_time