Online Coding Tests and Programming Assessments: How They Work

You click the link, a timer starts, and a problem statement appears next to a code editor. An online coding test can feel like a black box, but the mechanics are fairly simple: your code is run against test cases, most of which you cannot see, and the platform records how you worked. Once you understand how scoring and monitoring work, you can prepare for the test you will actually sit, not an imagined one.

This article explains how online programming assessments are built and scored, using the published documentation of two widely used platforms, Codility and HackerRank, and then walks through a worked example of the most common way candidates lose marks.

What an online coding test usually contains

Employers choose from libraries of task types. Codility’s description of its own task library shows the range: real-life skills tasks tied to specific languages and frameworks (front-end, back-end, DevOps, data science, QA and others), language-agnostic problem-solving tasks (algorithmic, bug-fixing, SQL, regex, multi-stage tasks), and multiple-choice technical knowledge questions.

In practice, a test for a developer role often mixes two or three of these:

• Algorithmic problems: write a function that solves a defined problem, such as finding pairs in an array or parsing a string.
• Bug-fixing: repair a short function that fails on some inputs.
• Practical tasks: build or extend a small component, endpoint or query.
• SQL: write a query that returns the expected rows.
• Multiple choice: language features, complexity, security or framework knowledge.

Read the invitation email carefully. It normally says how long you have, how many tasks there are, and whether a camera is required.

How scoring works

Test cases decide the score

Both platforms score code by running it against test cases. Codility states that “Generally, our tasks are scored on the basis of test cases” and that “Every task contains at least 6 test cases, often more.” The score is proportional: in Codility’s own example, failing 4 of 10 cases gives 60%.

HackerRank’s documentation describes the same principle from the test author’s side: each test case is marked as passed or failed, and the overall score depends on how many pass.

The examples you see are not the ones that count

This is the single most important thing to know. On HackerRank, test authors mark some cases as samples that you can see; every other case is hidden. In HackerRank’s words, “Hidden test cases are not visible to candidates”, and they exist to test edge conditions and unseen scenarios. Sample cases can be given a score of zero.

Codility is just as clear: tasks include an example test case “which is just an example and doesn’t count towards their overall score.” Its candidate FAQ even includes the question “Why did I score 0% despite passing example test cases?”

So when the editor says “OK” after you click Run, it means your code passed the visible example. It does not mean you solved the task.

Correctness and performance

Codility separates two kinds of test case. Correctness cases “measure how well the solution satisfies the requirements of the task”, including corner cases. Performance cases “measure how efficient the solution is” on large data sets, based on time complexity (Big O), and, according to Codility, apply only to its fundamental tasks. For failed cases, the candidate report shows what output was expected and what the solution actually returned.

That means a solution can produce correct answers and still lose marks because it is too slow.

Worked example: correct but too slow

Here is an illustrative task of the kind that appears in many assessments.

Task: Given an array of integers nums and an integer target, return the number of pairs of positions (i, j) with i < j where nums[i] + nums[j] == target. The array may contain up to 100,000 elements.

First attempt: nested loops

def count_pairs(nums, target):
    count = 0
    for i in range(len(nums)):
        for j in range(i + 1, len(nums)):
            if nums[i] + nums[j] == target:
                count += 1
    return count

This passes the example and every small test. But it checks every pair, which is O(n squared) work. With 100,000 elements that is roughly five billion comparisons, which will time out on a performance test case.

Second attempt: one pass with a dictionary

def count_pairs(nums, target):
    seen = {}
    count = 0
    for x in nums:
        count += seen.get(target - x, 0)
        seen[x] = seen.get(x, 0) + 1
    return count

Each element is processed once, with constant-time dictionary lookups on average, so the work grows linearly with the input. The answer is the same; the running time is not.

The edge cases hidden tests often check

Before submitting, run your own tests for inputs the example probably did not show:

• Empty array and single-element array (answer: 0).
• Duplicates, such as [2, 2, 2] with target 4 (answer: 3 pairs, not 1).
• Negative numbers and zero.
• Very large values, if your language has fixed-size integers.
• The maximum input size, to check speed.

assert count_pairs([], 4) == 0
assert count_pairs([4], 4) == 0
assert count_pairs([2, 2, 2], 4) == 3
assert count_pairs([-1, 5, 0, 4], 4) == 2

Most editors let you add custom input or write quick checks like these. Use them. Codility’s candidate FAQ includes a question on how to test your solution before submitting, which tells you the platform expects you to.

What the platform records about how you work

Modern assessments do not only look at your final code. Check the invitation and the platform’s candidate pages to see which features your employer has switched on.

HackerRank

HackerRank’s test integrity documentation describes several features: AI-based plagiarism detection that “flags potential plagiarism by analyzing candidate behavior, code evolution, and submission similarities”; image proctoring, where “the system captures periodic snapshots during the test” for the report; tab proctoring, which “tracks how often and for how long a candidate leaves the test window”; and copy-paste tracking, where “the system records the action automatically.”

Codility

Codility’s candidate report includes a similarity check indicator and a Timeline tab that replays the session: when you were active in the editor, when the window was out of focus, when text was pasted, when you ran verification and your final code. Codility notes that a paste does not automatically mean cheating, since candidates often write code in their own IDE and refine it in the editor. Codility also notes that reviewers can manually edit task scores.

What this means for you

• If the rules say no outside help, assume that pasting large blocks of code or long absences from the window will be visible.
• If you prefer writing in your own editor, check whether that is allowed first. Codility has a candidate FAQ entry on exactly this question.
• Commit to a language you know well. Reviewers may play back your session, and steady, readable progress looks better than frantic rewrites.

A two-week preparation plan

On the day

1. Read the whole problem, including constraints. The input size tells you whether a simple solution will be fast enough.
2. Write a correct solution first, then optimise if time allows. Partial credit is better than nothing.
3. Test the edge cases yourself before submitting.
4. Watch the clock and submit something for every task.
5. If something technical goes wrong, take a screenshot and email the recruiter straight away.

Coding tests usually come after a CV screen, so make sure your CV names the languages and frameworks you will be tested on. The software engineer resume builder and web developer resume builder include technical skills sections for this. Our separate software engineer interview questions guide covers the live technical rounds that follow.

FAQ

Can I use Google during an online coding test?

It depends on the employer’s rules, which are usually stated in the invitation or at the start of the test. Codility’s candidate FAQ has a dedicated entry on consulting a book, website or colleague. If the rules are unclear, ask the recruiter before you start.

Why did I score low when my code passed the examples?

Because example cases normally do not count. Hidden correctness and performance tests check edge cases and large inputs, so a solution that only handles the example, or is too slow, can score poorly.

Does code style matter in an automated test?

Automated scores are based on test cases. Codility’s candidate FAQ has an entry on whether coding style affects your score, so check it for your test, and remember that reviewers can read or replay your code. Clear names and simple structure help.

Which language should I use?

The one you are fastest and most accurate in, unless the role requires a specific language. Changing language under time pressure costs more than it saves.

Sources

• Automated scoring principles, Codility
• Reading a candidate test report, Codility
• Task library overview, Codility
• FAQ: Candidates, Codility
• Test cases in coding and approximate solution questions, HackerRank
• HackerRank test integrity, HackerRank