Key Takeaways
- Creyos scores are normed to first-time performance, so your first session is the natural baseline and small gains on repeats are expected.
- In Creyos’s own data, practice alone raised second-attempt scores by about 3% on average, with larger gains on some tasks such as Polygons and Rotations.
- Test-retest reliability averaged 0.69 but ranged from 0.45 to 0.92 by task, so some tasks are much steadier than others from one sitting to the next.
- The Meaningful Change Indicator uses a lenient threshold (p < 0.2); a flag means “unusual,” and no flag doesn’t prove nothing changed.
- There is no published evidence that repeat Creyos testing tracks medication response; a retest is one data point for your clinician.
Creyos, a computerized cognitive assessment that compares your task scores with people your age (see the Creyos guide), is built to be repeatable: the puzzles are generated fresh each time and the report tracks results across sessions. That makes retesting tempting, but repeat scores are easy to over-read. This page explains why the first session matters most, how much scores rise from practice alone, how to read the Meaningful Change flag, and what makes two sessions fair to compare. How often our clinicians repeat it is decided case by case; ask our team.
What repeat testing can and can’t do
What Creyos can and can’t do
It can
- Compare a later result with your own first result, task by task
- Flag changes larger than ordinary session-to-session variation
- Give your clinician a trend across several sessions
It can’t
- Prove that a treatment or medication caused a change
- Erase practice effects completely
- Replace how you are actually functioning day to day
Repeat testing is most useful when there is a specific question: Has something changed since a known starting point? It is least useful as routine checking without a reason, when ordinary fluctuations can look like news.
Why the first session is your baseline
Creyos’s normative database is centered on people’s first play of each task. Your first standard scores therefore compare like with like: a first-timer against first-timers. Every later session is compared with a norm that did not include practice, so a slightly higher score the second time is the expected result even if nothing about your brain has changed.
The report reflects this. Each task page shows a Meaningful Change Indicator against two reference points, your baseline (first) result and your most recent one. Once you have several sessions, a trend line is fitted through up to the seven most recent scores. Scores are always matched to your age norms on the date of each session, which matters for trends that span years.
A good baseline is one taken on an ordinary day: rested, not acutely ill, with the rules understood. A baseline taken during a crisis or after a sleepless week is still data, but later “improvement” may simply be recovery to your usual level.
How big are practice effects? Creyos’s own numbers
Creyos’s Science Overview reports, for each task, how well a person’s first and second scores agree (a Pearson correlation, r, where 1.0 would be perfect agreement) and how much the second score rose on average. The data came from people who chose to repeat tasks on the old public website, unsupervised, with 647 to 1,151 people per task. Task names are today’s; the table used older labels for some.
| Task | First-to-second agreement (r) | Average practice gain |
|---|---|---|
| Double Trouble | 0.92 | 4.90% |
| Grammatical Reasoning | 0.89 | 2.24% |
| Spatial Planning | 0.87 | 3.75% |
| Odd One Out | 0.73 | 1.55% |
| Rotations | 0.70 | 5.43% |
| Token Search | 0.66 | 4.99% |
| Digit Span | 0.64 | 1.33% |
| Spatial Span | 0.62 | 0.46% |
| Polygons | 0.60 | 7.91% |
| Number Ladder (formerly Monkey Ladder) | 0.57 | 1.62% |
| Feature Match | 0.57 | 4.09% |
| Paired Associates | 0.45 | −0.38% |
| Average | 0.69 | 3.16% |
Two lessons stand out. First, practice gains are modest on average but uneven: Polygons, Rotations, Token Search and Double Trouble rise most, while Spatial Span and Paired Associates barely move. Second, agreement varies widely. Double Trouble and Grammatical Reasoning are fairly stable from one sitting to the next; Paired Associates, Feature Match and Number Ladder are much noisier, so a single change on those tasks deserves extra caution.
An independent study points the same way. Kochan and colleagues had 52 adults aged 55 to 75 take a mix of Creyos (then Cambridge Brain Sciences) and Cogstate tasks at home, unsupervised, three times. Individual measures had reliability coefficients from 0.20 to 0.83, while combined global scores were above 0.8 over a one-month follow-up. Composites are steadier than single tasks.
New puzzles each time, but not true parallel forms
Many paper tests handle retesting with “parallel forms,” alternate versions with different items proven to be equally hard. Creyos takes another route: each task generates new problems automatically and adjusts difficulty to your performance, and Creyos says this minimizes learning effects. We found no formal parallel-forms study, though.
Fresh problems stop you memorizing answers, but they can’t stop you learning the game: understanding the rules faster, settling on a strategy for Spatial Planning, or feeling calmer the second time. That kind of familiarity is what the practice-gain column above captures.
Reading the Meaningful Change Indicator
The indicator uses a reliable-change method: it compares the difference between two of your results with the spread of differences seen when people in Creyos’s database repeat a task without any reason to change. Creyos has repeat data from large numbers of people, at intervals ranging from less than a day to more than a month. The thresholds are set separately for each task and each attempt number, and they are calculated from raw scores rather than the adjusted standard scores you see.
- The threshold is lenient. A change is flagged when a difference that large would occur by chance less than 20% of the time (two-tailed p < 0.2), far looser than the 5% level common in research. A flag therefore means “worth a look,” not “confirmed.”
- No flag isn’t proof of no change. Creyos’s own guide makes this point: a non-significant result only means the change wasn’t unusual by chance standards.
- Invalid scores are skipped. Meaningful change is calculated only when the task result passes the validity check.
- Practice still matters. For later attempts the bounds are lopsided: a larger rise than fall is needed to count, consistent with practice nudging scores upward.
For the rest of the report, including standard scores and the validity flag, see how to read a Creyos report.
How often to repeat it
We could not find a fixed retest schedule in Creyos’s public materials. The right interval depends on why you are retesting: confirming a flagged or surprising result, checking after a period of recovery (better sleep, an illness resolved), or comparing with baseline after a meaningful change in care. Testing again a few days later mostly measures practice; waiting too long lets life circumstances shift too.
At Willow & Stone, whether and when to repeat Creyos is a clinical decision made with you, and the retest interval and any fee are confirmed when you book or at your follow-up; ask our team. Follow-up visits themselves are listed on our pricing page.
Retesting around medication changes
People often ask whether a repeat test will show if a medication is working. There is no peer-reviewed evidence that Creyos tracks medication response, and its marketing claims to that effect rest on general studies of stimulant effects on similar tasks, not on Creyos-guided treatment. A clinician may repeat it to compare with your baseline, but any difference also carries practice effects, sleep, mood, stress and the timing of a dose.
That is why a retest is read alongside how you are functioning, symptom questionnaires and side effects. For more, see using cognitive tests around ADHD medication and when ADHD medication seems to stop working. Never start, stop or change a medication yourself because of a test score; talk with your prescriber first.
Making two sessions fair to compare
- Same device and input. Switching from a laptop with a mouse to a tablet, or to a trackpad, changes speed on timed tasks. (Phones are not recommended at all.)
- Similar time of day. Alertness changes across the day.
- Comparable sleep. Note how you slept the night before each session; poor sleep alone can lower some scores. Our sleep and cognitive performance page explains why.
- Same tasks. Compare a task only with itself; a different protocol is not a retest.
- Same medication routine on test day, as your clinician directs. Ask our team about caffeine and morning medication rather than guessing.
- Note anything unusual. A headache, a stressful day or an interruption is worth telling your clinician before results are interpreted.
Then treat the comparison as one input. Repeated measures are most informative when they sit inside measurement-based psychiatric care, alongside questionnaires and conversation. If a change shows up, the next steps are discussed in a visit, much like the planning described in after a psychiatric evaluation. Creyos is registered and listed with the FDA as a Class II, 510(k)-exempt computerized cognitive assessment aid; it is not FDA-cleared or approved and does not diagnose any condition.
Common Questions
Why did my Creyos score go up the second time?
Some rise is expected from practice alone. Creyos norms are based on first-time plays, and in Creyos’s own data second attempts averaged about 3% higher, with bigger gains on tasks like Polygons and Rotations. A higher score is only notable if it exceeds ordinary variation, which is what the Meaningful Change Indicator checks.
How often can you take a Creyos assessment?
The tasks generate new problems each session, so they can technically be repeated. How often it makes sense is a clinical decision: frequent repeats mostly measure practice. At Willow & Stone the retest interval is decided with your clinician; ask our team.
What does meaningful change mean on a Creyos report?
It means the difference between two of your results is larger than Creyos would expect from chance, using a lenient threshold where a change that size occurs by chance less than 20% of the time. It is a signal to look closer, not proof of improvement or decline, and the absence of a flag does not prove nothing changed.
Can retesting show whether my medication is working?
Not on its own. No peer-reviewed study has shown that Creyos tracks medication response. A clinician may compare a repeat result with your baseline, but practice effects, sleep, mood and dose timing all affect scores, so it is read together with how you feel and function.
Which Creyos tasks are least stable on retest?
In Creyos’s test-retest data, Paired Associates had the lowest agreement between first and second attempts (r = 0.45), followed by Feature Match and Number Ladder (both 0.57). Double Trouble (0.92) and Grammatical Reasoning (0.89) were the most stable. Changes on the less stable tasks need extra caution.
Part of our Creyos cognitive assessment guide
Sources & Further Reading
- Creyos — Science Overview (PDF), test-retest table
- Creyos — Interpreting Creyos Health Cognitive Assessment Results (PDF)
- Creyos — Understanding the Creyos Health Normative Database (PDF)
- Kochan et al. 2022, J Alzheimers Dis — remote unsupervised reliability (PMID 36314208)
- Creyos — ADHD Report Science Guide (PDF)
- eCFR — 21 CFR 882.1470, computerized cognitive assessment aid
This article is general education, not a diagnosis or advice for your situation. Creyos results support, and never replace, a clinical evaluation; only a clinician who has evaluated you can diagnose a condition or recommend treatment. Do not start, stop, or change a medication without talking to the clinician who prescribes it.
If you need help right now
Willow & Stone is an outpatient practice and is not an emergency service. If you or someone you care about is in immediate danger, call 911.
For urgent mental health support at any hour, call or text 988 to reach the Suicide & Crisis Lifeline, or text HOME to 741741 for the Crisis Text Line. Both are free, confidential, and available 24/7 nationwide.
Dr. Stacey Forbes, DNP, APRN, PMHNP-BC
Board-certified Psychiatric-Mental Health Nurse Practitioner and founder of Willow & Stone Integrative Mental Health. Nearly two decades of clinical experience; integrative, root-cause psychiatry in person in Arlington and by telehealth. Licensed in Texas, New Mexico, Florida, Alaska, Arizona, Colorado & Washington.
About Dr. Forbes →