A/B Test
Tests whether variants differ from the control on conversion rate (z-test) or mean (Welch's t-test), with lift, confidence intervals, p-values, verdicts and required sample size.
Analytics › Statistics Python visokio-agents
ABC Classification
Classifies items into A, B and C by cumulative share of value (Pareto), optionally within groups, plus XYZ demand-variability classes for ABC-XYZ inventory analysis.
Analytics › Supply Chain Python visokio-agents
Basket Affinity Matrix
Measures how often every pair of items is bought together (support, confidence, lift, Jaccard) from basket data, with each item's strongest partners.
Analytics › Retail Python visokio-agents
Benford's Law Test
Tests whether amounts follow Benford's law (first, first-two or second digit), overall or per group, with chi-square, MAD conformity and flagged records for follow-up.
Analytics › Audit Python visokio-agents
Case Duration and Bottlenecks
From an event log: case durations and rework, the slowest and costliest handovers between activities, and activity statistics.
Analytics › Process Python visokio-agents
Churn Survival Curves
Kaplan-Meier retention curves with censoring, confidence bands, median lifetime, milestone retention and log-rank tests between groups.
Analytics › Statistics Python visokio-agents
DBScan
Groups records into clusters by density (DBSCAN) on numeric fields and marks isolated records as noise. Outputs the input records with a Cluster field added: 1, 2, 3 and so on for clusters, 0 for noise.
Analytics › Clustering Python
Gaussian Mixture Model
Clusters records with a Gaussian mixture model on numeric fields, choosing the number of clusters itself, and gives each record its cluster and the probability that it belongs there. A second input can be assigned to the same clusters.
Analytics › Clustering R
KMeans
Groups records into a chosen number of clusters with k-means on numeric fields, standardised first. Outputs the input records with a Cluster field added.
Analytics › Clustering R
Cohort Retention
Builds acquisition cohorts from activity or order data and measures retention and revenue per cohort over time, with an average retention curve.
Analytics › Marketing Python visokio-agents
Concentration Measures
HHI, Gini, top-N and top-20% shares and Lorenz curves for customer, supplier or market concentration, per group.
Analytics › Finance Python visokio-agents
Contingency Table and Chi-square
Cross-tabulates two categorical fields with shares, expected counts and residuals, and tests their association with chi-square and Cramer's V.
Analytics › Statistics Python visokio-agents
Correlation Matrix
Pearson and Spearman correlations with p-values for all pairs of numeric fields or against a target, per group, ranked, with a heat-map matrix.
Analytics › Statistics Python visokio-agents
Cost Allocation
Apportions cost pools to departments, products or customers by drivers, with penny-exact largest-remainder rounding and a list of unallocated pools.
Analytics › Finance Python visokio-agents
Customer Lifetime Value
Historical and predictive customer lifetime value from orders (BG/NBD and Gamma-Gamma), with probability active, discounting and a holdout check.
Analytics › Marketing Python visokio-agents
Data Profiler
Profiles a dataset: one record per field with its type, missing and distinct counts, most common value and statistics, and the records with z-scores and anomaly flags for every numeric field.
Analytics Python
Find Nearest
Finds the k nearest points in a second input for each point in the first, by great-circle distance, with an optional distance limit and a catchment summary per destination.
Analytics › Geo Python visokio-agents
Forecast Accuracy
Compares forecasts with actuals per item and model: MAE, RMSE, MAPE, sMAPE, WAPE, bias and tracking signal, with flags, best model per item and a model ranking.
Analytics › Supply Chain Python visokio-agents
Funnel Analysis
Builds conversion funnels from event data: users reaching each ordered step within an optional time window, conversion, drop-off and time between steps, per segment.
Analytics › Marketing Python visokio-agents
Histogram and Distribution
Histograms with automatic bin widths shared across groups, and distribution statistics (percentiles, skewness, kurtosis) per group.
Analytics › Statistics Python visokio-agents
Holt-Winters Forecast
Exponential smoothing forecasts per series with automatic trend and seasonality selection, prediction intervals and a holdout accuracy check.
Analytics › Time Series Python visokio-agents
Inflation Adjustment
Restates money values in real terms of a base period using a price index table, per series, with the index used and cumulative inflation.
Analytics › Finance Python visokio-agents
Investment Appraisal
NPV, IRR / XIRR, MIRR, payback, discounted payback and profitability index per project from cash flows, with NPV profiles and multiple-IRR warnings.
Analytics › Finance Python visokio-agents
Likert Survey Summary
Summarises Likert-scale survey questions: answer distributions, mean scores, top-2-box, net scores and neutral shares, per question and segment.
Analytics › Survey Python visokio-agents
Linear Optimisation
Solves linear and integer programmes given as data (variables with objective and bounds, constraints in long format) with the HiGHS solver: optimal plan, slack, binding constraints and shadow prices.
Analytics › Prescriptive Python visokio-agents
Loan Amortisation Schedule
Generates payment schedules for loans and leases (repayment, equal principal, interest only, with balloon), monthly, quarterly or annually, with a summary per loan.
Analytics › Finance Python visokio-agents
Monte Carlo Cost Risk
Simulates three-point estimates and risk events to give P10 / P50 / P80 / P90 totals, contingency, the probability of exceeding a budget, and the drivers of uncertainty.
Analytics › Finance Python visokio-agents
Net Promoter Score
Net Promoter Score per group and period from 0-10 answers, with margin of error, confidence interval, weights, significance vs the rest and vs the previous period.
Analytics › Marketing Python visokio-agents
Attribute Analysis
From a list of connections (each record: a node, the node it connects to, and the first node's category), measures how much each category's connections lead to nodes it shares with each other category, such as which departments work on the same projects.
Analytics › Network Analysis R
TSNE
Places the nodes of a network on a 2-D (or 1-D or 3-D) map with t-SNE, so that nodes connected to the same other nodes end up close together and nodes with different connections far apart. Input: one record per connection.
Analytics › Network Analysis R
Occupancy From Intervals
Occupancy over time from start and end times: average and peak concurrency per bucket, utilisation against capacity and time over capacity, per group.
Analytics › Operations Python visokio-agents
Path Analysis
The most common event sequences per user or session, step-numbered transitions for Sankey charts, and next-step probabilities.
Analytics › Marketing Python visokio-agents
Period Comparison
Builds the standard month / QTD / YTD / moving annual total comparison versus prior periods per group, with a fiscal year start of your choice, plus a monthly trend table.
Analytics › Finance Python visokio-agents
Point in Polygon
Finds which polygon (territory, zone, area) each latitude / longitude point falls in, from GeoJSON or WKT shapes, and counts points per polygon.
Analytics › Geo Python visokio-agents
Point-to-Point Distance
Great-circle distance, bearing, direction and midpoint between two latitude / longitude pairs per row, with optional road distance and travel time estimates.
Analytics › Geo Python visokio-agents
K-Nearest-Neighbours
Predicts a category for each record from the k most similar records whose category is known (k-nearest neighbours), on numeric fields standardised first, with the share of neighbours that agree.
Analytics › Prediction R
Support Vector Machine
Predicts a category (such as Normal, Worn or Faulty) for new records from numeric fields, using a support vector machine trained on records whose category is known.
Analytics › Prediction Python
Price Elasticity
Estimates price elasticity of demand per product with a log-log regression (optional control drivers), with confidence intervals, revenue impact and demand curves.
Analytics › Pricing Python visokio-agents
Price Index
Laspeyres, Paasche and Fisher price indices, value and volume indices from item prices and quantities per period, per group.
Analytics › Finance Python visokio-agents
Price-Volume-Mix Bridge
Decomposes revenue change between two periods into volume, mix, price, new and lost product effects that add up exactly, per item and optional group.
Analytics › Finance Python visokio-agents
Principal Component Analysis
PCA on numeric fields: explained variance, loadings and correlations, automatic or fixed number of components, and scores per row.
Analytics › Statistics Python visokio-agents
Product Recommendations
Item-to-item recommendations from purchase history: top new products per customer with the reason, and similar products per product.
Analytics › Retail Python visokio-agents
RFM Segmentation
Scores customers on recency, frequency and monetary value from transactions and assigns named segments such as Champions, At Risk and Lost, with a segment summary.
Analytics › Marketing Python visokio-agents
Reconciliation Matcher
Matches two ledgers one-to-one on amount, reference and date with tolerances, plus group matches, and lists the unmatched items on each side.
Analytics › Finance Python visokio-agents
Regression
Linear or logistic regression with coefficients, standard errors, p-values, confidence intervals, odds ratios, VIF, fit statistics and scored rows.
Analytics › Statistics Python visokio-agents
Running Balance
Running balances per account from opening balances and dated transactions, with lowest balance, days below zero or a limit, and average daily balance.
Analytics › Finance Python visokio-agents
Safety Stock and Reorder Point
Calculates safety stock, reorder points, order-up-to levels and EOQ per item from demand history and lead times for a target service level, with a reorder list.
Analytics › Supply Chain Python visokio-agents
Sample Size Calculator
Sample size and power for A/B tests on proportions or means and for survey margins of error, per scenario, with days needed and power curves.
Analytics › Statistics Python visokio-agents
Seasonal Decomposition
Splits time series into trend, seasonal and residual components (additive or multiplicative) with seasonally adjusted values, seasonal indices and strength measures.
Analytics › Time Series Python visokio-agents
Stock Run-Out Projection
Projects stock day by day from on-hand, demand and open purchase orders to give run-out dates, shortfalls and what to expedite.
Analytics › Supply Chain Python visokio-agents
Survival Analysis
Estimates survival (or retention) curves from time-to-event data with censoring, by the Kaplan-Meier or Fleming-Harrington method, overall or per group, with 95% confidence limits and the median time per group.
Analytics › Statistics R
Text Word Frequency
Word and phrase counts in a text field, overall and per group, with stop words, n-grams and the terms most distinctive of each group.
Analytics › Text Python visokio-agents
Tiered Calculation
Applies tier tables (commission plans, tax bands, volume discounts) marginally or to the whole amount, per plan, with a breakdown by tier.
Analytics › Finance Python visokio-agents
Time Series Anomaly Detection
Flags spikes and dips per series against a robust rolling median and MAD baseline with weekday effects, centred or trailing.
Analytics › Time Series Python visokio-agents
Trade Area Summary
Counts and sums points within several radii of each site, with exclusive and shared catchment, and the nearest site and distance band per point.
Analytics › Geo Python visokio-agents
Model Validation
Checks a classifier's predictions against the actual values: a confusion matrix, and precision, recall and F1 score for each class with macro, weighted and micro averages.
Analytics › Validation Python
Variance Bridge
Bridges two totals (budget vs actual, year on year) by category as waterfall steps with favourable / adverse labels, per group.
Analytics › Finance Python visokio-agents
Website Analysis
Follows the links of a website from a start page and lists its pages with their title, heading, text and status, the links on each page, and the images.
Analytics › Website Python
Weighted Scoring
Multi-criteria scorecards: normalise criteria, apply weights, rank alternatives, apply must-pass thresholds and test how sensitive the winner is to the weights.
Analytics › Decision support Python visokio-agents
AI Chat - Claude
Sends a prompt from each record to an Anthropic Claude model and adds the reply as a Response field. Uses your own Anthropic API key.
Code & AI Python
AI Chat - DeepSeek
Sends a prompt from each record to a DeepSeek model and adds the reply as a Response field. Uses your own DeepSeek API key.
Code & AI Python
AI Chat - Google Gemini
Sends a prompt from each record to a Google Gemini model and adds the reply as a Response field. Uses your own Gemini API key.
Code & AI Python
AI Chat - Local LLM
Sends a prompt from each record to a language model on your own server, through an OpenAI-compatible endpoint, and adds the reply as a Response field.
Code & AI Python
Airtable
Reads every record of an Airtable table, or of one of its views, through the Airtable REST API, one record per row.
Connectors Python
Azure Data Lake Storage Gen2 Blob
Reads one CSV or Parquet file (blob) from a container in an Azure storage account, including Data Lake Storage Gen2, using the account's connection string.
Connectors › Azure Python
Google BigQuery Custom SQL
Runs a SQL query (GoogleSQL) on Google BigQuery and outputs the result, with BigQuery's types mapped to Omniscope field types.
Connectors › BigQuery Python
Google BigQuery Import Table
Reads a whole Google BigQuery table or view, page by page, with BigQuery's types mapped to Omniscope field types.
Connectors › BigQuery Python
Dune
Loads the result of a saved Dune query, public or your own: its latest stored result, or a fresh execution. Every page of rows is read, with numbers and dates typed from Dune's column types.
Connectors Python
Etherscan
Loads the native-coin balances, or the full transaction lists, of the Ethereum addresses in a field, from the Etherscan API (Ethereum and other chains by chain ID).
Connectors Python
FlightStats Airlines
Loads the list of airlines from the FlightStats (Cirium Flex) API, one record per airline with its code (FlightStats, IATA, ICAO), name, phone number and whether it is active. Needs a FlightStats App ID and App key.
Connectors › Flightstats R
FlightStats Airports
Loads the list of airports from the FlightStats (Cirium Flex) API, one record per airport with its codes (FlightStats, IATA, ICAO, FAA), name, city, country, time zone, latitude, longitude and elevation. Needs a FlightStats App ID and App key.
Connectors › Flightstats R
FlightStats Flights
Looks up each flight in the input (carrier, flight number and departure date) in the FlightStats (Cirium Flex) schedules and adds its scheduled airports, local departure and arrival times, aircraft type and codeshares. Needs a FlightStats App ID and App key.
Connectors › Flightstats R
Flipside
Runs a SQL query on Flipside's blockchain data tables through the Flipside API and writes every page of the result. Flipside has wound down its public query tools, so the API may no longer accept queries.
Connectors Python
HubSpot
Loads HubSpot companies, contacts, leads, deals, lists, forms, form submissions and associations into records, following every page, or searches contacts and companies.
Connectors › CRM Python
Jira
Loads the projects in a Jira Cloud site and the issues of a project or a JQL search, one record per issue with its status, people, dates, labels and components.
Connectors Python
AI Chat - OpenAI GPT
Sends a prompt from each record to an OpenAI model and adds the reply as a Response field. Uses your own OpenAI API key.
Code & AI Python
Overpass Street Coordinates
For each street name in a field, finds the OpenStreetMap streets with that name through the Overpass API, optionally inside a bounding box, and writes one record per point along each street with its latitude and longitude.
Connectors › Overpass Python
Salesforce REST OAuth
Loads Salesforce data through OAuth 2.0 and the REST API, with a simple object picker that automatically loads all available fields.
Connectors › CRM Python
Slack API WebClient
Calls any Slack Web API method, such as conversations.list or chat.postMessage, and turns the answer into records, following every page.
Connectors › Slack Python
Slim CD Transaction
Loads the transactions of a Slim CD payment gateway account (the SearchTransactions2 report), filtered by date range, customer, amount, card or transaction details, one record per transaction.
Connectors Python
Trello
Loads Trello boards, lists, cards, checklists and checklist items, and searches cards, with labels, members and due dates as fields.
Connectors Python
OpenWeatherMap
Adds the current weather, or a 5-day forecast in 3-hour steps, from OpenWeatherMap to each location in the input, found by latitude and longitude, city or postcode. Needs a free OpenWeatherMap API key.
Connectors › Weather R
XPT Reader
Reads a SAS transport file (XPORT version 5, .xpt): one dataset as records, plus a list of the datasets in the file and their fields.
Connectors Python
Yahoo Finance
For each record, loads the price history of the ticker in a field from Yahoo Finance: open, high, low, close, volume, dividends and splits per day, week, month or shorter interval, between two dates.
Connectors Python
Execute Command
Runs a program on the machine that runs Omniscope, once or once per record, passes the input records through and outputs each command's exit code, output and errors.
Code & AI Python
MongoDB
Reads the documents of a MongoDB collection, all of them or those matching a JSON query, one record per document.
Inputs › Databases Python
Snowflake Custom Query
Runs a SQL query on Snowflake, signing in with a password or a key pair, and outputs the result.
Inputs › Databases Python
Date Dimension
Generates a calendar table: calendar and ISO weeks, fiscal periods for any year start, UK or US public holidays, working-day counts and flags.
Inputs › Date Python visokio-agents
PDF Reader
Extracts the text of every PDF file in a folder, one record per page, with the file name and page number. Optionally writes a .txt file beside each PDF.
Inputs R
R Data Reader
Reads a table from an R data file (.rds, .rda or .RData): a data frame, matrix or vector saved from R. A second output lists the objects in the file.
Inputs › R R
Rds Batch Append
Appends many .rds files saved from R into one output: every .rds file in a folder, or the files listed in an input field. Files with different fields can be lined up first.
Inputs › R R
SFTP Downloader
Downloads the files in a folder on an SFTP server to the Omniscope server and lists them, ready for an Append files block to load.
Inputs Python
SharePoint Online Downloader
Downloads a file, or every file in a folder, from a SharePoint Online document library to the Omniscope server, ready for an Append files block to load.
Inputs Python
Web Form Session Login
Logs in to a website through its HTML login form, as a browser would, and outputs the session cookies and every response header, for later HTTP requests that need the session.
Inputs Python
Append PDF files
Combines the PDF files listed in the input records, in record order, into one PDF file, with an optional bookmark for each file.
Outputs › PDF Python
GitHub
Reads a CSV or other delimited file from a GitHub repository into records, or writes the input records to a file in a repository as a new commit.
Outputs › GitHub Python
Google BigQuery Export
Loads the input records into a Google BigQuery table, appending them or replacing the table's contents; the table is created if it doesn't exist.
Outputs › BigQuery Python
Multipart Form Upload
Posts files to an HTTP endpoint as multipart/form-data, one request per file, and records each response's status and body.
Outputs › API Python
Report tab to PDF
Prints a Report tab to a PDF file for each input record, from the tab's address in a field, saving the files in a folder.
Outputs › PDF Python
Multi-tenant Report to PDF
Prints one Report tab once per scenario of a multi-tenant configuration, saving a PDF file per scenario in a folder.
Outputs › PDF Python
Report to PowerPoint
Takes a screenshot of each Report tab in the input and makes a PowerPoint file with one slide per tab.
Outputs › PowerPoint Python
Send Report to Slack
Takes a screenshot of the Report tab at each input record's URL and posts it to a Slack channel with the record's message and the link; optionally also one PDF of all screenshots.
Outputs › Slack Python
Slack Bot
Posts one message to a Slack channel for each input record, as your Slack app's bot, optionally with a file attached.
Outputs › Slack Python
Send to Telegram
Sends the input records as a message to a Telegram chat, group or channel through your bot, as text, a table or JSON.
Outputs › Messenger Python
Web Image-PDF output
Saves a screenshot (PNG) of the web page at each input record's URL, at a chosen size, and optionally one PDF with a page per screenshot.
Outputs › PDF Python
Add row ID field
Adds a field numbering the records 1, 2, 3 and so on, in the order they arrive, working through the data in chunks so large inputs never have to fit in memory.
Preparation Python
Age and Tenure
Exact age or service in years, months and days, decimal years, bands, next anniversary and upcoming milestones, with 29 February handled.
Preparation › Date Python visokio-agents
Anonymise
Replaces names, emails and other text with generated code words, and shifts numbers and dates by random noise, to make a safer copy of a dataset for demos and testing.
Preparation Python
Business Hours and SLA
Elapsed business hours between timestamps within opening hours, excluding weekends and holidays, with SLA due times and met / breached status.
Preparation › Date Python visokio-agents
Canonical Schema Mapper
Maps the many names a field goes by (aliases) to one canonical name, combines fields that hold the same thing, and converts each to a set type with an optional default, following rules kept as data (typed into the block or in a CSV file), working through the data in chunks so large inputs never have to fit in memory.
Preparation Python
Conditional Execution
When its yes/no option (usually linked to a project parameter) is on, starts a separate run of the listed blocks of a project; when it is off, does nothing.
Preparation › Workflow Python
Data Cleansing
Removes empty rows and fields, fixes missing values, strips unwanted characters and changes case in one step, with a report of what changed.
Preparation › Cleansing Python visokio-agents
Data Compare
Compares two versions of a dataset by key: added, removed and changed records, field-level differences with tolerances, and schema differences.
Preparation › Audit Python visokio-agents
Data Quality Analyser
Finds duplicate records, missing values and numeric outliers, and outputs an issue log, the data annotated with its issues, and a copy with missing values filled.
Preparation Python
Data Validation Rules
Validates data against a rules table or typed rules (not empty, unique, ranges, lists, patterns, emails, postcodes, field comparisons) with violations per row and per rule.
Preparation › Data quality Python visokio-agents
Date Range Expander
Expands each start / end date range into one row per day, week, month, quarter or year, with days in each period and amounts prorated by days.
Preparation › Date Python visokio-agents
Duplicate Survivorship
Finds duplicates within a dataset on several match keys, keeps a survivor per group by rules and fills its gaps from the duplicates.
Preparation › Data quality Python visokio-agents
Expand Date Fields
Adds year, quarter, month, week, day and day-name fields for each chosen date field, for filtering and grouping by part of a date without formulas.
Preparation Python
Field Renamer
Renames the fields of a data set from a list of rules (current name, new name) held in another data set, and lists the rules that matched nothing and the fields left unchanged.
Preparation R
For Each (Separate Workflows)
Runs another Omniscope project once per input record, setting its project parameters from the record before each run, and lists the result of every run.
Preparation › Workflow Python
For Each Multi Stage
Runs another Omniscope project once per input record in up to three stages, each a separate run of its own blocks, with an optional stage before the loop; sets the project's parameters from each record first.
Preparation › Workflow Python
Project Parameters Batch Setting
Sets project parameters in one or more Omniscope projects: each input record gives a project's address and the values for parameters named like its fields.
Preparation › Workflow Python
Centroids from GeoJSON
Finds a centre point (latitude and longitude) for each shape, from records with GeoJSON or WKT shapes or from a GeoJSON file or shapefile, and outputs it with the shape's ID or other fields.
Preparation › Geo Python
Gridsquare
Converts Maidenhead grid square locators (such as IO91wm) to the longitude and latitude of the square's centre.
Preparation › Geo Python
Shapefile
Finds which region each latitude / longitude point falls in, and adds the region's label to the point's record. The regions come from records with GeoJSON or WKT shapes on a second input, or from a shapefile or GeoJSON file.
Preparation › Geo Python
UK Postcode Tools
Validates, standardises and extracts UK postcodes and splits them into area, district and sector with area name and country, offline.
Preparation › Geo Python visokio-agents
Imputation
Fills missing values with the mean, median, mode, a constant, previous / next value or interpolation, optionally within groups, with indicator fields and a summary.
Preparation › Cleansing Python visokio-agents
Kedro
Runs a Kedro project's pipeline on the block's inputs: each input is saved to a named dataset of the project's catalog, and up to two datasets the pipeline produces become the block's outputs.
Preparation › Interfaces Python
JSON Expand fields
Turns a text field holding a JSON object into one new field per key, next to the input fields; nested objects and arrays become JSON text.
Preparation › JSON Python
JSON Normalise
Turns JSON text into a flat table: each object becomes a record and each array item a record of its own, nested objects become fields such as customer.name, and the input fields are repeated on every record.
Preparation › JSON Python
As-Of Join
Joins each row to the record in force at its date (latest on or before, next, or nearest) per key, with a maximum gap and validity end dates.
Preparation › Join Python visokio-agents
Fuzzy Terms Join
Joins two inputs on text fields that are written differently but share words, such as company names or product descriptions. Each record is paired with the record on the other side that shares the most distinctive words.
Preparation › Join R
Fuzzy Match
Finds near-duplicate records within one input (with group IDs), or the best matching record in a second input, using fuzzy text similarity.
Preparation › Join Python visokio-agents
Inequality Join
Joins the first and second inputs where a number or date field of the first is less than, greater than or equal to a field of the second (<, <=, >, >=, ==), optionally only where another field is equal as well. Every matching pair is output.
Preparation › Join R
Interval Join
Joins each record of the first input to the intervals in the second input that contain its value (a number or a date between a start and an end), optionally only where another field matches as well.
Preparation › Join Python
Markdown to HTML
Converts Markdown text in a field (bold, lists, links, headings, tables) to HTML in a new field, keeping the other fields.
Preparation Python
Multi-Field Formula
Applies one operation (round, scale, share of row or column total, index to base, z-score, fill, text case...) to many fields chosen by list, name pattern or type, with optional row statistics.
Preparation › Formula Python visokio-agents
Multi-Row Formula
Window calculations across rows within groups: previous / next value, change, % change, running total, rolling mean / sum / min / max over rows or days, rank and share of group.
Preparation › Formula Python visokio-agents
Name Parser
Splits full names into title, first, middle, last and suffix, tidies capitalisation, flags organisations and builds salutations.
Preparation › Parse Python visokio-agents
Outlier Flagger
Flags low and high outliers in many numeric fields at once, within groups, by IQR, z-score or robust MAD, with bounds and optional capping.
Preparation › Data quality Python visokio-agents
Partition
Numbers records in consecutive groups of a chosen size (1 for the first 1,000 records, 2 for the next 1,000 and so on) in a new Partition field, for processing or sending data in batches.
Preparation › Partition Python
Melt De-pivot
Keeps the chosen fixed fields and turns every other field into records: one record per input record and field, with the field's name and its value. Values of different types become text.
Preparation › Pivot Python
RegEx Parse
Applies a regular expression to a text field: parse capture groups into fields, replace with back-references, tokenise matches into columns or rows, or flag matches.
Preparation › Parse Python visokio-agents
Rolling Time Window
Trailing time-window aggregates per key (count, sum, mean, min, max, distinct) for several windows, such as transactions per card in the last 24 hours.
Preparation › Window Python visokio-agents
Sequence Gap Finder
Finds missing, duplicate and out-of-order numbers in invoice, cheque or ticket sequences, per series, with a completeness summary.
Preparation › Audit Python visokio-agents
Sessionise Events
Groups each user's timestamped events into sessions by an inactivity gap, adding session IDs to events and producing a session summary with duration, landing and exit events.
Preparation › Date Python visokio-agents
Set Project Parameters
Sets project parameters, often of the project it is in, from records holding a parameter name and a value; all values are set in one request.
Preparation › Workflow Python
Smart Date Parser
Parses text fields holding dates in mixed formats (ISO, day-first or month-first, named months, compact, with time zones) into Date fields, and flags the values it could not read.
Preparation Python
Smart Schema Normaliser
Cleans up an export with untidy field names: makes the names lower case with underscores, merges fields whose names differ only in case, spaces or dashes, and gives each text field a number, date, yes/no or text type, working through the data in chunks so large inputs never have to fit in memory.
Preparation Python
Sort fields
Reorders the fields of the data by name (case-insensitive, with numbers in names in numeric order: Q2 before Q10) or by type, optionally keeping chosen fields first.
Preparation Python
Split Address
Splits a street address such as "Kerkstraat 12A" or "221B Baker Street" into the street name, the house number and a suffix (the letter or addition after the number).
Preparation Python
Standardise
Rescales the chosen number fields to the range 0 to 1 (min-max scaling): each field's lowest value becomes 0, its highest 1, and the rest fall proportionally between.
Preparation › Standardisation R
Stratified Sample
Draws a reproducible random sample within strata: proportional with a minimum per stratum, equal, fixed number or percentage, with sampling weights and an allocation table.
Preparation › Sampling Python visokio-agents
Streaming Field Renamer
Renames the fields of the input from rename rules (current name, new name) written in the block or kept in a CSV file, working through the data in chunks so large inputs never have to fit in memory.
Preparation Python
Subtotals and Rollup
Aggregates by a hierarchy of fields with subtotal rows at every level and a grand total, shares of parent and total, in report order.
Preparation › Aggregate Python visokio-agents
Tile
Assigns records to tiles of a numeric field: quantiles, equal-sum Pareto bands, equal width, standard-deviation bands or manual cut points, optionally within groups.
Preparation › Binning Python visokio-agents
Time Duration Unit Converter
Turns times of day and durations into a number of milliseconds, seconds, minutes or hours, in a new field for each chosen field, ready to add up, average or compare.
Preparation Python
Time Series Filler
Inserts missing days, weeks, months or other periods in each time series and fills values with zero, the previous value or interpolation, with a gap report.
Preparation › Date Python visokio-agents
Top N With Other
Keeps the top N items per group by value and rolls the rest into an Other row, with ranks, shares and cumulative shares.
Preparation › Aggregate Python visokio-agents
URL Encode
Percent-encodes the text in a field (UTF-8) into a new field, so it can go into a link, a query string or a web request.
Preparation Python
Unescape HTML
Turns HTML character references such as &, é and £ in a text field back into the characters they stand for, in a new field.
Preparation Python
Unstack Records
Splits text fields that hold several values in one cell, such as one item per line, into one record per value, repeating the other fields.
Preparation Python
Working Days
Counts working days between dates and adds working days to dates, skipping weekends and UK or US public holidays, with SLA checks.
Preparation › Date Python visokio-agents
About these blocks and the visokio-agents tag (74 of the 163 carry it).