Knowledge Is Power!

Managing Knowledge As Staff+

“Throughout my career, I’ve forgotten many times more than I know now.”

Me, Numerous Occasions

Yes, software engineering is a job that exposes us to a ton of knowledge. Yet, at any given moment, we only use a subset of it. It may not be that challenging when we are working on one project at a time. This usually allows us to rely on a specific subset of our knowledge for a prolonged period and gives a space to familiarize/refresh a different one when the time to change a project comes. But for staff+ engineers that’s rarely the case. We can be hit with a random challenge at any given time and are expected to be somewhat knowledgeable about it. This is why we need a way for constant management of an ever-growing set of knowledge. It can’t be done in our heads (although you might believe that at the beginning of your path), it has to be digital and at the end of our fingertips.

I’ve been building and evolving my approach to managing knowledge throughout my entire career, although I can’t say I was always doing it intentionally. For quite a long time I was reading books, attending conferences, and gaining experience from projects and it simply “stayed with me” (or at least I thought so). I believed that my knowledge was growing just fine until on one occasion I faced a challenge that I’ve seen before, solved before, and didn’t remember how to do it again. That was the moment I understood that I needed to store my knowledge outside my head. Since that moment I’ve explored multiple tools and approaches to manage my knowledge externally. I’m going to share with you my current approach. I’ll show you the tools I’ve found the most useful and the way I’ve made them “mine”.

But before I start, there is one more thing I’ve learned over time that is crucial for knowledge management from staff+ perspective - you don’t only curate knowledge for yourself but also for others. As staff+ engineer, you’re not only responsible for managing knowledge that you may need, but you also need to actively manage knowledge that others in the organization will need. And those are two separate tasks.

Managing Knowledge for Yourself

Managing knowledge for yourself is mostly about reuse and future use. You’ve just solved a problem or finished a research and you feel that you will reuse that new knowledge in the future. Or you have just stumbled upon a new piece of information (maybe an excerpt in a book or blog post you’ve just skimmed through) that you don’t need right now but might in the future. You need a way to provide this knowledge to your future self. You need a knowledge vault.

Knowledge vault is a structured approach to retaining your knowledge over time. You can find many ready-to-use systems for building a knowledge vault, but the truly valuable ones are always personalized. My current system for the knowledge vault has its origins in the Second Brain system described by Tiago Forte in his book “Building a Second Brain”. Tiago proposes PARA as a way to organize your knowledge. PARA focused on putting knowledge into four categories biased toward action:

• Projects (short-term, something you are working on right now)

• Areas (long-term, something you want to manage over time)

• Resources (topics of interest)

• Archives (inactive, past knowledge)

Those categories have proven to work for me almost exactly as defined, although my approach to Archives is a little bit mixed (more about that in a moment). What completely didn’t work for me was the proposed digital representation - folders in a notes app. It felt awkward for me and I felt pulled into more familiar tooling. This is why my Projects and Areas categories live in GitHub. You read it right - GitHub is the main tool behind my Second Brain. For Projects, I create a GitHub repo and a project. The same for Areas. That doesn’t mean what I keep in there is mostly code. I store every kind of resource there and GitHub nicely handles the majority of them. The only category I don't keep in GitHub is Resources - for that I use… OneDrive 😉. This may sound like quite an odd choice of tools, but for me, it works great, especially when it comes to the activities that the Second Brain system proposes for managing your knowledge. Because just having a structure is not enough to be able to say that you are managing your knowledge. First of all, the information must find its way into your vault.

My tools allow me to capture new information easily. I have both GitHub and OneDrive available on all my devices and whenever I feel I need to capture something I can either create an issue or drop something into a folder. But there is a trap here. That ease of capturing information can lead to capturing too much, which will make your knowledge vault useless. Here Second Brain comes with a single advice - “Keep What Resonates”. Some additional questions that attempt to help you further like “Does It Inspire Me?”, “Is It Useful?”, “Is It Personal?”, or “Is It Surprising?”, but ultimately it boils down to “Keep What Resonates”. Sadly, I don’t have any better advice here. I agree that this is the best possible advice although becoming proficient at it takes time and practice. Luckily that practice can be done through follow-up activities. After all, capturing an information often doesn’t make it valuable for future you, it might require distilling.

What you need to do to distill the knowledge in your vault will depend on you and the nature of the knowledge. In my case, the knowledge I capture from experience (solved problems, research, etc.) usually doesn’t require further distilling. But with all the glimpses of information from other sources, it’s quite different. One step is always extracting “metadata”. Those “metadata” (together with my repositories and OneDrive folder structure) are the cornerstone of discoverability. I don’t do anything fancy here - just some keywords in names. For the Resources that’s usually it - at some point they will either move into suitable Projects or Areas. For the knowledge in Projects and Areas, I will go further. I will rewrite the original issue to be more descriptive, I will start linking to external resources, and I will be bringing additional artifacts (and tying back the commits to the issue for discoverability). If the knowledge is technical, I may end up implementing some samples to make that knowledge truly actionable for the future me (still tied to the original issue). And in the end, everything will end in Archives.

I’ve already mentioned that my approach to Archives is mixed. If something spends a lot of time unused in the Resources it simply gets deleted. In the case of Projects and Areas - nothing disappears from git 😉. I will close the issue and delete content from the repo, but there is always a way to dig it up.

This is how I manage knowledge for myself and it doesn’t have to be the right way for you. I strongly encourage you to read about Second Brain and other systems for building a knowledge vault (an interestingly looking one is “The CTO Notebook”), pick the one that looks attractive to you, and then make it your own. Here you don’t have to make any compromises, which is not the case when it comes to managing knowledge for others.

Managing Knowledge for Others

Managing knowledge for others is mostly about the here and now. It’s also about ease of access. The goal is to provide a repository for your engineering organization that enables efficient access to knowledge needed daily. The answer is an internal knowledge base.

Your responsibility, as a staff+ engineer, is not to build the internal knowledge base. Of course, you will be one of many people building it, but primarily your responsibility is to be its custodian. That starts with deciding what kind of materials to put into your organization's knowledge base. As with your personal knowledge vault, putting too much quickly makes the knowledge base useless as it becomes hard to find what people are looking for. That’s why you should put a proper top-level structure into your knowledge base. Extending that structure shouldn’t be to use and every category in the structure should have its guardian - a person maintaining the integrity of the category (sadly, it’s likely that you will be the one for multiple of them). What categories should you have? That will strongly depend on your organization’s needs, but certainly, there will be a category for onboarding materials, engineering organization policies, and internal technical documentation. The last one is usually the most challenging and unique to engineering organizations as it often escapes the main tool used for the knowledge base.

The reason technical documentation often escapes the main tool used for the knowledge base is that it’s best kept together with whatever it’s describing (library, API, or service) - in a repository. This way the documentation can be maintained together with what it describes. Engineers can be mandated to update the documentation as part of their workflow. This includes ADRs, diagrams, API descriptions and other kinds of documentation. Does it mean that the knowledge base itself should be only a catalog of links? Ideally not. Ideally, we should have automated Continuous Documentation pipelines that build and push the updated technical documentation to the internal knowledge base. This gives us three benefits. The first is that the knowledge base is the organization’s body of knowledge and everyone can be referred to it instead of several different repositories. The second is that engineers can use a more “technical” format for the documentation (from my personal experience I can say that Markdown and Mermaid get the job done) while API descriptions can be generated directly from the code (of course if we need to grow a culture of properly documenting APIs in code 😉). The third is that we increase the documentation freshness (and nobody likes outdated documentation).

Yes, the freshness of the documentation is important - not only the technical one. This is why every material in the knowledge base should have an owner. Of course, just assigning an owner doesn’t help. We need at least two processes to make that valuable. The first process is the review, which has two forms. One is the usual “change” review - when someone is modifying some material, the owner should review that change. The other is periodic review. At a given cadence (from experience I would suggest a couple of months to a year) the owner should take a look at the material and decide if it’s still valid in its current form, or maybe it needs an update, or has expired and should be removed. This way you can help your internal knowledge base stay up to date. At this moment some of you are probably asking “Who remembers to do that” and the answer is “Almost no one” 🙂. That periodic trigger for review should be automated.

If we are automating so much, and a significant part outside of the main knowledge base tool, does the tool we choose matter? Yes, mostly from the perspective of familiarity. The ease of access that I’ve mentioned at the beginning of this section comes from familiarity with the tool (because the search capabilities suck in all of them - this is the space where I’m hoping for AI to bring some positive change). This is why we should choose the tool that is most popular within companies of similar profile to ours. One popular choice is Confluence. Another, mostly from Microsoft bubble, is Sharepoint. There are also others like Notion, Guru, or Document360. My advice? The same goes for the engineering tools for the technical documentation. You need to have a good reason to not use Git as your repository. I’ve also already mentioned Markdown and Mermaid, which from my experience are the most popular, but in your team that can be AsciiDoc and Structurizr (or maybe PlantUML). In general - don’t be original, go for familiarity.

In Conclusion

Managing knowledge is hard, and depending on who you do it for it’s hard in different ways.

Doing it right for yourself is hard because it’s intensive. You need to be constantly on the lookout when it comes to capturing and you need to be constantly diligent when it comes to distilling.

Doing it right for others is hard because it’s boring. You need to be constantly patient when it comes to finding the right compromise around the scope and you need to be constantly persistent when it comes to ensuring freshness.

But in both cases it’s crucial for long-term success and you should never neglect it.