unnest()no longer makes a field of an empty row or entry. A dynamic matrix nobody filled in arrives as a list of empty rows from SurveyJS 3.2.4 on, and a dynamic panel always kept its empty entries; both became fields likeq.0holding{}. Now they give no field at all, as in the R package.
Needs platform 0.114 or later.
responses()brings the columns of the participants' panel, as the platform's data files do:panel(the panel's name, with one panel per cohort the cohort) and your own columns aspanel_<column>, for examplepanel_patient_code(first name, last name and e-mail address only for an identified study). They come with the respondent identifier;con.last_panel_columnsnames them.waves()keeps them once per person instead of one copy per wave (panel_patient_code, notpanel_patient_code_wave1,panel_patient_code_wave2). A value one wave gives, such as the name from an identified wave, is not lost to another wave that does not have it.- A text question for a number (input type Number or Range) is numeric in the codebook now, also as a matrix cell or a field of Multiple Textboxes, so it arrives as a number.
snapshot()withoutsincefailed with "The since field must be a valid date" (422): it sent an emptysince, which the platform refused. It now leaves it out; platforms from 0.110.1 on also accept an empty one, so 0.9.0 works there too.
Play a study's AI interviews from a script (platform 0.107, needs a token with the permission "Run simulations").
components()lists what can be simulated, each with its parameters —defaultis the value the study has now.start_simulation()begins one with your own parameters and inputs;Simulation.respond()answers the current stimulus with text, a value or a summary choice. A model call that fails records nothing on the platform, so the same response is sent again.- A question the interview answers with buttons (
stimulus["expects"] == "choice") takes one of the values it offers, as in the interview:respond(value=..., other=..., comment=...), orrespond(skip=True)wherequestion["can_skip"]. Text is not taken there. The stimulus lists what the buttons offer (choices,special_items,rate_values, …). simulate()plays a whole simulation with a function of yours as the respondent.Simulation.attach()keeps your own data with it;simulations()lists them again, by run, component or status.- A used-up daily budget raises
BudgetExhaustedat once, withretry_after, instead of waiting until midnight.ApiErrornow carries the platform'sreason. examples/: prompt optimisation with GEPA, and a plain evolutionary search.
Same as the R package 0.10.0.
scale(data, score="sdq_emo")works out a scale score the questionnaire computes itself (Σ in the editor): the codebook now says which rows it counts - those of its subscales, in a loop those of the same copy -, mean or sum, and how many have to be answered. That is the way to give responses recorded before a score was added or changed the score of now; the platform keeps every response's score as it was computed then. Needs a platform that describes its scale scores (version 0.106 or later); with an older one, name the items.scale()takesmethod="sum"as well as the mean, and gives whole numbers as the platform writes them (4, not 4.0) for rows.- The codebook carries
subscaleon a matrix row andscale_scoreon a scale score.
options()gives one field per option of a multiple-choice question, a tag box, a picture choice or a ranking, as the export files have them: 1 when the option was chosen, 0 when the question was answered without it, None when it was not answered; for a ranking the option's place. In a frame the columns are integers that can be missing (Int64).unnest()adds those fields right after the list (options=Falseleaves them out).
The tests now include a study with multiple choice ("None", "Other" with its text), a tag box, a ranking, a rating, yes/no, a drop-down with "Don't know", text boxes, a constant sum with its total, an SDQ-like matrix with subscales and the questionnaire's own scale scores, and a loop with a score per copy - recorded from the platform, and checked value by value against the platform's own export file. The platform's scale score cases now include sums.
con.files(survey_id) downloads the files of a study's "File Upload" questions and unpacks
them: a folder per question, the files named by response (drawing/1234-1.png) rather than by
the name a participant gave them, and a list with one row per file - whose it is
(response_id, and respondent_id where your token may read it), the original name, type,
size, checksum and local_path, where the file now is. Join it to responses() over
response_id.
In the responses themselves an uploaded file now appears as its name, type, size and a reference; the file is no longer in the row.
verify(data, certificate)checks a certified dataset — a CSV file the platform issued with a.certificate.json— and returns aVerification: whether every certified cell is what participants entered, which survey, organisation and issuer it belongs to, which columns and rows were left out and which columns are in the file without being certified — appended ones, and a removed column put back.print()it for a summary; it is true only when the file is valid.- The file is read strictly, exactly as the platform reads it: UTF-8, quotes as in RFC 4180, every row as wide as the header. A file two programs could read differently — a stray quote, a lone carriage return, a NUL, a row with more fields than the header — cannot be verified.
- The summary sorts the columns of the file as the platform's verify page does, each in one
place: certified, in the file but not certified (left out when the certificate was issued,
computed by the platform, or added afterwards) and missing from the file;
Verification.column_groups()gives the same as lists. It also names the values the participant's browser reported and the responses collected offline. - Only
https://sondavi.iois trusted unless you name another installation withtrust=. The keys ofhttps://sondavi.iopublished when this version was made come with the package, so a certificate signed with one of them is checked without a connection. Any other key is fetched from the issuer, or given withkeys=to check offline. - A certified file — your own CSV, Excel or JSON file, compared by the platform with the
stored responses or with a file certified earlier — is checked by its fingerprint, whatever
its format: valid when it is that very file, modified otherwise.
result.basisand the printed summary say how the platform read it, how many responses of the survey it holds, which were left out (incomplete, screened out, by group), which columns are your own, which are written differently, and which certificate it was derived from. - To share fewer columns or rows, or another format, certify the changed file on the platform that issued the certificate; that works even after the responses were deleted there. Deleting columns or rows by hand, or opening the file in a spreadsheet program and saving it again, makes the certificate fail.
- Still the standard library alone — the signature check is part of the package.
The package checks itself against certificates the platform itself issued, so the two cannot read a file differently unnoticed.
scale() now gives the same numbers as the scale score the platform computes in the
questionnaire ("Add scale score"). Two cases differed:
- An item marked "Reverse-scored" whose scale has no numbers (columns named
column1,column2…) was counted as answered, unmirrored. It now counts as unanswered, with a warning naming it — its reversed copy is empty in the platform's export, too.items()gives it as None. - A
min_itemsabove the number of items gave None for every row. It now means every item.
The package checks itself against the platform's own cases, so the two cannot drift apart unnoticed.
- Each response now carries
response_status:"complete","partial"(stopped before the end) or"screened_out". Inframe()it is a Categorical with all three categories, in this order, sovalue_counts()shows a 0 where nobody was screened out. - A status this version does not know is kept as a category of its own, with a warning.
- The client now names its own version when it talks to the platform (it said 0.4.0).
- A study can now screen people out (the platform's "Screen out participant" rule). Such a
person is a row like an unfinished one -
completed_atis None - and carriesscreened_out_atandscreen_out_reasoninstead.screened_out_atarrives as adatetimelikecompleted_at, also inframe(). - A bot check asked as a question arrives as a field of its own,
"passed"/"failed"(inframe()with its labels), andbot_check_passedis filled for every study that checks.
The tests now include a study with a screen-out and a bot check question, recorded from a real instance.
unnest()writes an AI interview question the way the platform's export and codebook do:<question>.status,.turns, and where they apply.answered,.skippedand.summary. It used to spread the stored conversation into one column per message and field (talk.messages.3.text…), and the columns the codebook names for the question were missing.conversations(rows)is the conversation as a long table, one row per message: who wrote it, the respondent's answer number, the text, typed or spoken, and when.
The tests now include a study with AI interviews, recorded from a real run.
scale(data, names)is the scale score per row,items(data, names)the items as numbers (a DataFrame forcorr()or a reliability analysis). Name the items as in the questionnaire: a reverse-scored one is taken from its reversed copy by itself. Works on a frame (which carries its codebook) and on rows (passcodebook=con.codebook(42)).- They read the numbers from the codebook. In a frame the items are Categoricals, whose codes are positions: a "Don't know" offered as sixth option was a 6.
- By default every item has to be answered;
min_itemsallows gaps on purpose.
The tests now also compare the reversed copies with the platform's own export file of the same study, value by value.
On the platform you can mark a matrix row as "Reverse-scored": an item worded the
other way round within a scale. The answer is stored as given, and the platform's
export files add a reversed copy <name>_r = (smallest + largest scale value) −
answer.
unnest()now adds the same field, right after the item it mirrors: a number, computed from the answer as given, and None for an answer off the scale such as "don't know". Name and arithmetic are the platform's, sob5.worry_rmeans the same in an exported file and in data read through the API. It needs the codebook: passcodebook=con.codebook(42)with rows; a DataFrame fromframe()brings its own.- The codebook's variables carry
reverse_scored: {"min", "max"}for a marked row.
This needs a platform that sends reverse_scored in the codebook; with an older one
there is simply no copy.
unnest(con.frame(42)) spreads the nested columns of a frame in place and gives the
new columns the codebook's treatment, as frame() does for the questions at the top
level: a matrix cell with value labels becomes a Categorical, a numeric one a number.
frame() keeps its codebook in df.attrs["sondavi_codebook"] for this;
codebook=[] leaves the cells as stored. frame(42, unnest=True) gives the same
result.
A value the codebook has no label for — an answer to an option deleted after people
had chosen it, a code added later without a label — used to become NaN in frame().
NaN reads as "not answered", so the answer was lost without notice. It is now kept as
a category of its own, named by the value, after the labelled ones, and a warning
names the column and the values.
A numeric variable holding something that is no number still has NaN there, but now with a warning saying what was dropped.
- In
frame(), three kinds of column came out entirely NaN: yes/no questions (the codebook writestrue/false, the answers were compared asTrue/False), numerically coded questions with at least one missing answer (pandas makes such a column float, and4.0did not match4), and multiple-choice questions (a list compared as text). The first two now read as their labels; a multiple-choice answer stays a list, as in the R package. unnest()dropped nested fields whose name starts with__. They are part of the stored answer and are now kept, as the R package always did.
The rows now also carry the platform's internal fields under their own names
(__locale, __randomization_order, …), and a question named like one of the row's
own fields (source, locale, completed_at, …) arrives as <name>_answer. The
package passes both through unchanged. Server-side change — your platform needs
updating for it.
The features match the R package 0.4.0.
The platform has a new question type: participants paint areas or set pins on an image. Its answer arrives as the stored object — the image, the grid and the cells or pins.
markings()turns these into one row per painted cell or pin, with the position relative to the image and in pixels of the original — the same table asmarkings.csvin the platform's image-markings export. This is what a heatmap is drawn from.unnest()writes them the way the CSV export does: one field per marking type, cells as row runs ("2:3-5 3:4"), pins as"x,y"pairs. Before, it spread the stored object intomap.image.src,map.grid.cols, … — names the codebook does not know.
Same as the R package 0.3.0.
Test infrastructure only; the package itself is unchanged.
The run now picks a free port instead of a fixed one, and checks that the answer on it comes from the fixture server rather than merely that something is listening — a foreign service passed the old check, and the tests then ran against it and hung. When the server does not come up, its output is printed instead of a bare timeout.
Functional parity with the R package.
completed_at and started_at now arrive from responses() as datetime objects in
the platform's own time zone, which the API states in its response. They used to be
strings, and frame() was the only place they became dates.
This was worth a breaking change because the old behaviour failed silently: strings
sort lexically and compare against a datetime without complaining. Pass
parse_dates=False if you need the raw strings, e.g. to json.dumps() the rows.
The time zone uses zoneinfo only when the platform runs on something other than UTC,
and falls back to naive datetimes if the zone database is unavailable — on Windows that
would otherwise mean depending on tzdata, and this package depends on nothing.
unnest()spreads matrices and dynamic panels into one field each, using exactly the names the platform's own export writes — so an analysis built on an exported file and one built on the API agree on variable names. Also asframe(42, unnest=True).Connection.waves()joins the waves of a study series on the respondent identifier. Everyone seen in any wave is kept, so the people who stopped answering stay in the table instead of being dropped by an inner join.Connection.fingerprint()returns the line that belongs next to a published result.snapshot_responses()now warns when responses have been deleted since the snapshot was recorded, and separately when their contents have changed. A replay with fewer abilities than the snapshot was recorded with stays quiet — the digests cannot be compared there, and a warning that fires when nothing is wrong teaches you to ignore the one case it exists for.
- The Windows test run died with a
UnicodeEncodeError: the console is cp1252 and the tick in the output is not in it. The run no longer goes through a shell script either, so it is the same on every system. - The package carried the operator's own institute as the name of the platform. It is called Sondavi wherever it runs.
First release: connect(), surveys(), codebook(), responses(),
iter_responses(), frame(), and snapshots for citable datasets.