Human skills rise with practice: a 2,630-person study

The skills in this study go by several names. Researchers and workplace publications increasingly call them human skills, especially in the context of the AI era and what humans bring that machines do not. Practitioners often still call them soft skills, the older umbrella term. Happily calls them Power Skills, the developmental targets in our feedback loop. They are the same six capabilities under different labels: Critical Thinking, Self-awareness, Optimism, Leadership, Initiative, and Empathy. We use Power Skills below for consistency with our prior research.

The framework defense in our last article argued these six are worth coaching toward. This one asks the harder follow-on question: when people actually use Happily over time, do their skill ratings move?

The honest answer takes two corrections to get right. The first is selection. We can only study people who stuck around long enough to accumulate enough ratings, and those people are different from a random employee in ways we cannot perfectly correct for. The second is regression to the mean. People who start low tend to rise just because that is where there is room, even without any coaching. A clean read on coaching effect has to put both effects in the same picture and only claim what survives them.

What survives is a positive dose-response signal that is independent of starting level. Heavier users of Happily gain more rating points than lighter users, in every starting-rating quartile we look at. The effect is largest where there is most room to grow, and smallest at the ceiling. That is the cleanest finding the data can support, and the rest of this article walks through how we got there and what it does, and does not, mean.

What we see, after the corrections

Mean change across the whole cohort is positive at +0.29 points, and 67% of person-skill cells show a positive change. That is the headline, but it pools two distinct effects: a real dose-response signal and a regression-to-mean artifact. The chart below pulls them apart.

Two patterns sit in this chart. Read the chart along its vertical axis to see regression to mean. People who started in the bottom quartile (dark purple) sit highest on the chart, even at low engagement, because the lowest starters have the most room to rise. People who started in the top quartile (lightest line) hover near zero across all engagement levels, because they are at the ceiling and have nowhere to go. The vertical stack of the four lines is the regression-to-mean signal.

Read the chart left to right within any one line to see dose response. Every line slopes upward from the low-engagement column to the high-engagement column. Within every starting-level quartile, heavier users of Happily gained more rating points than lighter users. That is the part of the picture that regression to mean does not explain.

Pulling the two effects apart

The unstratified gap between top-quartile and bottom-quartile engagement is +0.35 points. After controlling for starting level by averaging the gap within each starting-level quartile, it falls to +0.21 points. About 40% of the raw difference was regression-to-mean leakage. The remaining 60% is the genuine dose-response signal.

The dose effect is positive in all four starting-rating quartiles, but the magnitude narrows as starting level rises. People who started in the bottom quartile gained nearly a full third of a rating point more when they engaged heavily with Happily than when they barely engaged. People who started near the top of the rating scale show only a +0.07 dose effect, which is small but still in the right direction.

Per skill: where the signal is strongest

Start with the picture. For each skill, the chart below traces the rating journey of the most-engaged quartile, in the skill's colour, against the least-engaged quartile, in grey, from the first half of each person's window to the second. On every skill, the most-engaged moved further. For Initiative, the least-engaged group did not move at all.

Two things are tangled in that picture. The most-engaged groups tend to start lower, so part of their larger gain is room to grow rather than engagement itself. Separating the two means controlling for starting level.

Averaging the stratum-controlled dose effect across the four starting-level quartiles gives one number per skill. All six are positive. The three skills with the largest stratum-averaged dose effects are Self-awareness, Initiative, and Optimism. The smaller effects on Critical Thinking, Empathy, and Leadership are still real, but tighter.

Critical Thinking shows the largest unstratified gain (+0.43), but only an average stratum-controlled dose effect (+0.15). That pattern is consistent with strong baseline-level recovery in the lowest quartile but smaller dose-response. Initiative shows the opposite: a modest unstratified gain (+0.21) but the second-largest dose effect (+0.28), pointing to a steadier link between engagement and rating change.

What the study shows

This study is observational and dose-comparative. Among people who use Happily for at least six months, those who engaged more saw larger rating changes than those who engaged less, and the gap holds after controlling for where they started. That dose-response pattern is the strongest evidence an observational study can offer that the loop contributes to the change. A randomised comparison, with engagement as the only difference between matched groups, would measure that contribution directly, and is the natural next step for this line of research.

What this gives us is a falsifiable, defensible empirical baseline. If the framework were inert, the dose-response gap would collapse inside starting-level quartiles. It does not. It narrows, in the direction the ceiling predicts, but stays positive at every starting level and on every skill. That is the empirical anchor for the case that the loop does something.

Limitations

• The cohort is self-selected. To enter the analysis, individuals had to accumulate at least six ratings on the same skill over at least 180 days. That selects for engaged users at sticky companies. The result is not a population-level claim about employees; it is a claim about people who use Happily this way.
• Observational design. The dose-response comparison is the strongest lever available short of a randomised trial. It does not by itself separate the loop from unmeasured common causes, such as a person whose role, manager, or organisational context is changing in ways that also lift ratings; a controlled comparison would isolate that.
• The AI rating model evolves over time. If the model became more generous between 2024 and 2026, observed gains would partly reflect model drift rather than skill change. A future iteration of this study should freeze the model version across the analysis window.
• Score-rating coupling. The rating is calibrated on text, and people who write more text give the model more signal to work with. Heavier users write more, which may make their ratings both more accurate and slightly inflated. The size of this effect is not estimated here.
• The ceiling at the top quartile (mean Δ near zero) is a real measurement constraint, not a finding about that group. We need a finer-grained scale or a different outcome (e.g. specific competency observations) to measure development among already-strong individuals.
• Per-skill comparisons should be read with the sample sizes in mind. Leadership has 1,322 cells and Self-awareness has 2,234. Smaller-cell skills will have wider confidence intervals than this article reports.