29 The In-Package Assistant
The external-agent path has the package acting as an engine and you as the planner. This chapter covers the other arrangement: the package also writes the plan, by asking a language model what to do.
That convenience is real, and so is its price. sclet needs aisdk, a configured provider, and a network call for every plan. The chunks here are therefore gated on SCLET_BOOKDOWN_EVAL_AI rather than on the package alone: on a machine with a provider configured they execute and the outputs below are real, and on a machine without one — a CI runner, most of them — they are skipped. Rendered prose that depends on a model’s mood is not a regression test, and a build that quietly spends credits is not a build.
29.1 Requirements
## [1] "deepseek:deepseek-chat"# Everything else works without any further setup. `NULL` here means the
# default, which is to inject the shipped skills; `getOption()` is used rather
# than `options()` because the latter evaluates an unset name and errors.
getOption("sclet.ai.use_skills")## NULLoptions(sclet.ai.use_skills) controls whether the shipped skills are injected into the planning prompt. It is on by default, and the AI foundations chapter explains why the skills — not the inline prompt — carry knowledge-type rules such as stage ordering.
29.2 An object to ask about
The examples below need an object with clusters to find markers in, and — for the audit section — an integration that exposes a corrected expression layer. Building both here keeps the chapter self-contained:
set.seed(42)
demo <- scuttle::mockSCE(ncells = 200, ngenes = 60)
demo <- SingleCellExperiment::SingleCellExperiment(
assays = list(counts = SummarizedExperiment::assay(demo, "counts")),
colData = S4Vectors::DataFrame(SummarizedExperiment::colData(demo))
)
demo$batch <- rep(c("donor_1", "donor_2"), each = 100)
demo <- NormalizeData(demo)
demo <- FindVariableFeatures(demo)
demo <- ScaleData(demo)
demo <- RunPCA(demo, ncomponents = 10)
demo <- FindNeighbors(demo, dims = 1:10)
demo <- FindClusters(demo, resolution = 1.0)
demo$ident <- as.character(SingleCellExperiment::colLabels(demo))
length(unique(demo$ident))## [1] 529.3 AIPlanAnalysis: ask for a plan
## [[1]]
## [[1]]$id
## [1] "step_1"
##
## [[1]]$action
## [1] "run_de_test"
##
## [[1]]$params
## [[1]]$params$name
## [1] "cluster_markers"
##
## [[1]]$params$min.pct
## [1] 0.1
##
## [[1]]$params$logfc.threshold
## [1] 0.25
##
##
## [[1]]$depends_on
## character(0)
##
## [[1]]$description
## [1] "All-cluster marker path (no ident.1/ident.2/group) yields a marker table per cluster in one round; prerequisites logcounts and colLabels are both present."
##
## [[1]]$expected_outputs
## list()
##
## [[1]]$requires_confirmation
## [1] TRUE
##
## [[1]]$max_retries
## [1] 0
##
## [[1]]$continue_on_error
## [1] FALSEYou will see a warning on the way, and it is the privacy layer doing its job rather than a problem:
sclet AI outbound payload requires user consent before use. Fields requiring consent were redacted: active_states.clustering, features.hvg.selected
Those two fields are the clustering and the selected variable features. Both are results of the analysis, not inputs to it, so they are held back unless you say otherwise — the point of the ledger is that a model can plan the next step without being handed the answer. The plan above is still correct because it does not need them: “find markers that distinguish the clusters” is a goal, and the cluster labels are read locally at execution time. If a later round does need the feature set, pass privacy_consent = TRUE to the call that makes it, and know that you have chosen to share it.
The result is an ordinary plan — the same structure you wrote by hand in the previous chapter — so it is validated, previewed and executed by exactly the same functions. AIPlanAnalysis() also spends one repair round against validation errors and goal coverage before returning, so what you get back has already been checked once. plan$metadata$goal_coverage$missing names goal items with no step behind them; a hand-written plan gets no such help, so check it yourself.
29.3.1 The registry has to match the goal
AIPlanAnalysis() plans against every action group by default. RunAIAnalysis() differs: it defaults to a beginner registry — read, preprocess, dimred, graph, cluster — which is the right set for “run a pipeline from raw counts” and the wrong one for “find markers”, because run_de_test lives in the annotation group.
What that costs you is worth watching once, because the failure is instructive rather than silent. With the beginner registry the planner is shown only its eleven actions, so for a marker goal it proposes a step it was never offered, and validation refuses the plan:
action is not registered: run_de_test; it exists, but in the 'annotation' registry group, which this registry did not request -- add 'annotation' to include
Two things are true at once here. The registry you pass is part of the question — pass the groups your goal needs. And the engine’s refusal tells you exactly how to fix it, because a bare “not registered” would have sent you looking for a typo.
29.4 RunAIAnalysis: plan, preview, ask, execute in one call
result <- RunAIAnalysis(demo, goal = "find markers that distinguish the clusters",
include = c("read", "preprocess", "dimred", "graph", "cluster", "annotation"),
confirm = "yes")
result$status## [1] "completed"## [1] "completed"RunAIAnalysis() is one round: plan, validate, preview, ask, execute, and optionally interpret. Running it again is how you continue — there is no loop option to look for, for the reason given in the previous chapter: the engine executes one plan and never re-plans mid-run.
result$status is the execution outcome; result$report$status is the planning outcome. They are different fields because they are different questions, and a plan can be fine while an execution fails.
Passing confirm = "yes" executes without pausing, which is right for a script and wrong for exploratory use. Leave it out and read the preview first.
29.5 Free-form questions with sclet_copilot
Not every question is a plan. sclet_copilot() answers a free-form question about an object by reading its recorded provenance — what analysis ran, in what order, with which parameters — instead of looking at the data alone.
sclet_copilot(demo, "I clustered this object after normalization. What should I check before trusting the clusters?")## [1] "Read-only status readout of the sclet dataset plus a proposed descriptive-spine plan; nothing was executed. The dataset is 200 cells x 60 genes with counts/logcounts/scaled layers, a 10-PC PCA, and a k=10 rank SNN graph (200 vertices / 9184 edges). The recorded chain (normalize_logcounts, pca, knn_graph, louvain_clusters) is completed. colLabels has 5 groups (46/47/45/46/16), all above the 10-cell aggregation threshold. No marker test, no trajectory, and no reference annotation have been run; velocity, differential abundance, spatial and multimodal questions are blocked by missing inputs/records. Proposed plan: (1) cluster-vs-rest marker test on logcounts, (2) Treatment contrast (cell-level only, no replicates), (3) optional trajectory ordering on PCA + colLabels, which is not a lineage claim."The value here is narrow and worth stating. The tool cannot see your cells; it can see what the ledger recorded and in which order. That is enough to discuss the workflow, and not enough to make a claim about the biology.
29.6 Cross-chain auditing
AuditAnalysisChain() asks a narrower question: does the expression of specific genes look distorted by an earlier step? It traces the state machine back to the raw layer and compares.
It needs an integration that produces a corrected expression layer. fastMNN does, but only when asked for full-gene output — by default it returns a corrected embedding and the logical corrected layer falls back to the source expression, which is not something to audit:
demo <- RunIntegration(demo, method = "fastMNN", batch = "batch", correct.all = TRUE)
# The logical layer is the vocabulary; the assay behind it is an implementation
# detail that the audit resolves for you.
SummarizedExperiment::assayNames(demo)## [1] "counts" "logcounts" "scaled"
## [4] "reconstructed"audit <- AuditAnalysisChain(
demo,
features = head(rownames(demo), 5),
raw_layer = "counts",
integrated_layer = "corrected"
)
cat(audit)## The ledger establishes the object's structure (200 cells x 60 genes; assays counts/logcounts/scaled/reconstructed; layers counts/logcounts/scaled/corrected/reconstructed; 5 colLabels groups) and that of the five requested features only Gene_0003 is a selected HVG, while Gene_0001, Gene_0002, Gene_0004 and Gene_0005 are non-selected bystanders carried into `reconstructed` only under `correct.all = TRUE`. It does not establish whether the requested features agree between `counts` and `reconstructed`: no per-gene layer statistics are recorded, and no ai_evidence nodes exist in the supplied ledger. The chain also has structural problems that constrain interpretation: the HVG step requested n = 5000 on a 60-gene object and selected 29 genes without reconciliation; the knn_graph and Louvain clusters were built on uncorrected PCA (dims 1:10, layer `scaled`, integration = null) while the active reduction is the 28-component `corrected` embedding, so the clusters are not batch-corrected clusters; the corrected embedding has no recorded downstream use; integration has two records with a flagged conflict and fastmnn lineage parents are null; `counts` was never corrected and never used for the graph; design semantics (batch, Treatment) are unconfirmed; and there are no markers, trajectory or annotation. Verification is proposed as four steps: a layer-agreement check for Gene_0001-Gene_0005, re-deriving the graph on the corrected embedding, a marker test on the resulting clusters, and confirmation of design semantics before any condition contrast.What it returns matters more than what it prints. The results are recorded with claim levels, and the report’s own verification section re-derives them; a round that is not rendered loses that check. Audit output is input to a judgement, not the judgement.
Be aware of which integration method you used:
fastMNNproduces a corrected expression layer (withcorrect.all = TRUE), so the comparison above is the right abstraction.HarmonyandscVIproduce a corrected reduction — no corrected expression exists to compare against, so this particular audit does not apply and needs adapting.
29.7 A plan does not replace the workflow functions
RunBasicWorkflow(), RunStandardPipeline() and the other Run<Mainline>Workflow() shells are the way to run a whole mainline in one call, and they are not registry actions: a shell chains several actions, and a plan that used one would have a single opaque step where the engine wants inspectable ones. Plan the same work from the actions it is made of — that is what the trace records, step by step, and it is why a completed round can be read back as a graph rather than as one black box.
29.8 What the planner can and cannot see
The in-package assistant plans against the same registry as everything else, but it is shown a subset: an action whose required input only a person can supply — a gene set, a bulk phenotype matrix, a reference object — is marked planner_visible = FALSE and stays out of its catalog. RunAIAnalysis() therefore covers the analyses whose inputs live on the object, and a goal that needs external material is a driver-mode job, covered in the driver chapter.
That distinction is worth knowing before you blame the model. A goal about gene-set scoring is not a capability the assistant lacks; it is a goal it cannot plan, and the honest response is to say which input is missing rather than to propose a step that would fail validation.
29.9 What the package will not do
It will not call a provider unless you ask for one explicitly; it will not present a claim stronger than its evidence supports; and it will not treat a successful plan as a successful analysis, because “all steps completed” and “the question was answered” are different statements. The last chapter covers how that difference is enforced.