FeaturedSocialEngineeringTechnical

The Question That Built Our Engineering Grading System: Would I Trust This Person On-Call?

8 Mins read

Elena Galinovskaya, Team Lead of an L3 support team at SOFTSWISS Casino Platform, spent four years watching promotion decisions create more confusion than clarity. This article draws on her direct experience building an engineering grading system from scratch – including the mistakes, the fixes, and the metrics that made the difference.

When grading criteria exist only in a manager’s head, every promotion decision invites disputes over subjective assessments and erodes team trust. Transparent grading systems do not replace managerial judgement – they structure it around the measurable impact an employee has on the team.

Grading Starts with Your Expectations

Before implementing complex matrices, a manager needs to clarify their own expectations. Ask yourself: “What exactly do I want to delegate to someone at this level?” The answer to that question determines what skills are required.

For the engineer, this translates into: “What do I need to demonstrate in order to progress?” Once expectations are spelled out, promotion stops feeling like a black box – engineers can see exactly what they need to do and plan for it. This puts an end to the corrosive comparisons that quietly undermine team cohesion: “Why did my colleague make Senior when I ship more work than they do?”

The Competency Matrix: From Intuition to Structure

To stop expectations from staying vague, translate them into a skills matrix. It organises requirements into areas:

  • technology,
  • product knowledge,
  • process knowledge,
  • soft skills.

A simple example:

JuniorMiddleSenior
TechnologyTerminal commands for working with filesAbility to write simple Terminal scripts: executing commands, saving results, using loops, conditions, and sleep
Product KnowledgeUnderstanding which third-party integrations the product uses
Process KnowledgeUnderstanding the areas of responsibility of different teamsAbility to identify problems in the solution delivery process
Soft SkillsBasic ability to present results, plans, and questions at daily stand-ups clearly and conciselyAbility to deliver presentations on a given topic at internal knowledge-sharing sessionsAbility to coach team members on presentation skills

Some cells are left intentionally blank – at certain grades, a given dimension may not yet be relevant or is assessed through other means.

The matrix should never become a simple technology checklist. Knowing a tool exists is not the same as being able to architect a solution around it – the matrix needs to reflect that difference explicitly, both in depth of proficiency and in how much weight each skill carries.

There is also a pattern worth noting: the lower the grade, the greater the focus on technical skills; the higher the grade, the greater the emphasis on leadership and soft skills. The Middle grade typically sits in between – technically solid, with enough soft skill to work independently and push back when needed.

The practical value of the matrix:

  • For hiring: Interviews become consistent across all candidates. Hiring managers assess every candidate against the same criteria, instead of making intuitive comparisons between incompatible profiles.
  • For development: Engineers identify their blind spots and understand where to focus their efforts.
  • For the business: Leadership gains clear visibility into where team expertise is lacking. This is exactly the pattern we found in our own team – solid framework knowledge across the board, but a critical gap in infrastructure maintenance that wasn’t visible until we mapped it.

What a Senior Is: Going Beyond the Role

Defining the Senior level is where most grading systems break down – not because the concept is complex, but because it requires the manager to admit what they actually need from someone, rather than what sounds good on a job description. A common mistake is to treat Senior simply as “a very experienced specialist”, and that framing will not hold up under scrutiny.

A Senior is someone who makes the work of the entire team more predictable and stable. Before we had someone filling that role explicitly, incidents moved from discovery to resolution through whoever happened to be available – there was no clear ownership. Whereas Junior and Middle engineers are responsible for their own output, a Senior is responsible for the conditions that make everyone else’s output possible.

In practice, a Senior:

  • Sees not just the bug, but the process defect that caused it.
  • Aligns with adjacent teams before conflicts arise.
  • Relieves the manager by taking on mentorship and acting as the technical authority in disputes.
  • Keeps the team moving during incidents or escalations when the lead is not available. Someone needs to own the call, and it should not always fall to the manager.

In effect, a Senior is the manager’s right hand. With a Middle engineer, you need to define the task. With a Senior, you name the problem and step back.

Drawing on Global Experience: Senior as a Systemic Role

When I was defining the Senior level, I kept coming back to what mature engineering organisations had already worked out. This is not a definition we invented from scratch – it is consistent with how high-performing engineering teams think about the role, and each of these frameworks directly shaped criteria we ended up using.

  • Technical leadership (Staff Engineer model): The Staff Engineer model draws a hard line between people management and technical leadership – a Senior is not a junior manager. This distinction directly shaped how we framed the role: we made it explicit in the matrix that a Senior is assessed on how much they increase the team’s autonomy and decision-making quality, not on whether they manage headcount.
  • Eliminating toil (Google SRE): This framework maps directly onto L3 support engineering. A Senior does not simply work through the queue – they change the system so the same tickets stop coming back.
  • Performance metrics (DORA): A Senior engineer’s impact should show up in the numbers – MTTR, change failure rate, the ratio of reactive to proactive work. In an L3 support context, these are not abstract; they are the numbers you see every week. We used this framing to add a measurable component to the Senior assessment: candidates needed to show they had moved a relevant metric, not just contributed to tasks.
  • Blameless culture (Etsy): Running a proper retrospective is part of the job at this level. The goal is not to find someone to blame – it is to understand how the system failed and what changes will prevent a repeat. We made retrospective quality an explicit Senior criterion: not “has the engineer attended retrospectives”, but “has this engineer led one that produced a systemic change”.

Confirming the Senior Grade: Assessing Facts, Not Words

We learned quickly that a promotion process only builds trust when it is grounded in evidence. Impressions from a recent conversation are not enough – people remember every case where the decision felt arbitrary. Rather than relying on those impressions, we moved to a promotion packet.

To advance to a higher grade, a specialist must demonstrate not potential, but actual impact on the project:

  • Examples of complex incidents they coordinated.
  • Improved processes or documentation standards they introduced.
  • Successful measures to reduce or optimise a category of recurring tasks.
  • Successful examples of mentoring colleagues.
  • A visible reputation among peers – colleagues go to this person first when things get complicated.

An important principle: a promotion must confirm behaviour that has already become habitual. If someone needs a title in order to start taking ownership, that is a risk. A genuine Senior begins to influence the team well before any official appointment.

Early Mistakes: Challenges We Encountered

Building a functioning grading system is always a process of trial and error. Here is what we at SOFTSWISS got wrong before we got it right – five lessons from building this inside our own team.

  • Testing theory rather than practice on the product. When assessing candidates for the Middle level, we initially evaluated only general theoretical knowledge. Without practical tasks set within the real product architecture, this produced false results: engineers memorised terminology but operated as Junior specialists in practice. We added sections covering product knowledge and team processes to address this.
  • Updating the matrix without notice. When skill requirements changed quietly, managers found themselves in a difficult position. An engineer who had fully met every criterion in the previous version was entitled to a promotion, even if those criteria were already outdated. We learned to announce changes in advance and set a clear transition window – typically one review cycle – so engineers were not penalised for criteria that shifted under them.
  • Running assessments in the same order every time. Sessions always followed the same sequence. Early modules received thorough coverage; by the final topics, interviewers had little energy left. Rotating the module order, introducing timers, and bringing in new interviewers significantly improved consistency.
  • Re-assessing Junior engineers from scratch. We were running full assessments for every Junior hire – the same process we used for Middle candidates. It was not scalable and told us less than the hiring interview already had. A more effective approach is to map the gaps identified at the hiring stage and close them through focused assignments and targeted check-ins over the first review cycle.
  • Selecting Senior engineers on technical knowledge alone. Previously, candidates went through an in-depth interview only with the development teams. Soft skills and professional reputation were not assessed. As a result, each person became a Senior by their own criteria, which were opaque to everyone else. Bringing in a broader panel and assessing soft skills explicitly gave us consistent, defensible outcomes – and meant candidates knew in advance exactly what they were being evaluated on.

Real Results: What Was Achieved in Practice

Addressing these mistakes and establishing transparent rules of engagement delivered measurable results for both the business and the teams.

  • Protected the team from talent attrition. Having already defined requirements for Junior and Middle, we could move quickly when it came to Senior. This reduced the risk of engineers leaving due to unclear growth prospects – a pattern that had driven departures in previous years. The department gained its first official Senior engineer, who has already taken meaningful workload off managers and built a process for resolving the team’s immediate problems.
  • Gained the ability to measure department-wide competency. Linking the skills matrix to Jira tasks let us collect real metrics. We could see, for example, that three people on the team had strong application-layer skills but nobody owned infrastructure reliability – a gap that nobody had flagged because nobody owned it, and that only became visible when we mapped the matrix against Jira task history. Managers now have visibility into skill levels not just for individuals, but across the team as a whole.
  • Made engineer growth plannable. Engineers began proactively building their own development plans because the steps became tangible. One engineer on the team – someone who had been hovering at the Middle level for over a year – ran through the full preparation programme and passed the assessment within a month of it launching.
  • Created an environment for knowledge sharing. Preparing for a promotion is no longer something engineers do alone. They began openly consulting their managers, technical experts, and Senior colleagues – particularly after we brought Senior specialists in to conduct assessments.
  • Scaled the practice across the company. Other departments picked up the approach and applied it in their own teams. The Talent Development and Culture team drew on our matrix when building unified grading standards across SOFTSWISS – something I did not anticipate when we first put it together.

How to Start: the Iteration Principle

When I started, I wrote down the answer to one question: what would I need to see from someone before I felt comfortable delegating an on-call shift to them? That single question generated the first version of our matrix.

Start where I started: one question about what you actually need from someone at that level. The answer is your first draft. From there, iterate:

  • Start with a simple list: What is critical right now?
  • Run a pilot: Try evaluating one or two people using this system.
  • Gather feedback: Is it clear to engineers what is expected of them?
  • Adapt: If a criterion is not working or is redundant – remove it.

When people know what is expected of them and can see the path forward, they stop wasting energy on anxiety about their standing – and start spending it on actual work.

About the Author

Elena Galinovskaya is Team Lead of an L3 support team at the SOFTSWISS Casino Platform. She has worked in IT for over five years, the last four at SOFTSWISS, where she progressed from L3 developer to team lead. Elena combines day-to-day team management with technical leadership and internal project work in a PM capacity. She is an active participant and speaker in SOFTSWISS’ internal management community. 

Leave a Reply

Your email address will not be published. Required fields are marked *