- Multiple Python versions are handled correctly (#1444)
- JSONC fixes (#1426)
- New documentation design
- ASV supports using JSONC as a configuration file (#1420)
- Update deprecation for latest conda versions (#1403)
- Fix runs for profiles commit results (#1404)
- Internal pip parser now works for git URLs with branches and tags (#1431)
asv
defaults to--force-reinstall
now to facilitatecompare
andcontinuous
for manually versioned projects (#1421)
build
is now the default backend forasv
to install projects. (#1387)
- Partially skipped benchmarks will still have their results displayed. (#1351)
asv
will now correctly prepare all the build backend dependencies intobase_requirements
and the defaultbuild_command
has been modified to allow fetching from PyPI. (#1377)
- The
asv
package no longer prepends the script execution directory unconditionally. Now we check for and remove the path only if it matches the directory that the runner script resides in. (#1346) - The
bdist
wheels no longer includebenchmarks
andtest
. (#1349) - The
mamba
plugin works correctly for newer versions (>=1.5) oflibmambapy
(#1372) - The
mamba
plugin respects theMAMBARC
environment if set, taking channels and channel priority from the file in the environment variable. (#1373) - Fixed a bug where
matrix
requirements were dropped if an environment file was specified. (#1373) conda-forge
is no longer a default channel formamba
. (#1373)
asv
now depends onvirtualenv
(#1379)
pip
dependencies inenvironment.yml
files for themamba
plugin are handled correctly (#1326)asv.config.json
matrix requirements no longer needpip+
set explicitly for calling thepip
solver forvirtualenv
asv
will now useconda_environment_file
if it exists (#1325)
asv
timestamps viadatetime
are now Python 3.12 compatible (#1331)asv
now providesasv[virtualenv]
as an installable targetasv
now uses Github Actions exclusively for Windows and Linux
asv_runner
is now used internally, making the addition of custom benchmark types viable (#1287)- Benchmarks can be skipped, both wholly and in part using new decorators
skip_benchmark_if
andskip_params_if
(#1309) - Benchmarks can be skipped during their execution (after setup) by raising
SkipNotImplemented
(#1307) - Added
default_benchmark_timeout
to the configuration object, can also be passed via-a timeout=NUMBER
(#1308) ASV_RUNNER_PATH
can be set from the terminal to test newer versions ofasv_runner
(#1312)
- Removed
asv dev
in favor of usingasv run
with the right arguments (#1200) asv run
andasv continuous
don't implement the--strict
option anymore, and they will always return a non-zero (i.e.2
) exit status if any benchmark fail.
- Fixed
install_timeout
forconda
(#1310) - Fixed handling of local
pip
matrix (#1312) - Fixed the deadlock when mamba is used with an environment file. (#1300)
- Fixed environment file usage with mamba and recognizes default
environment.yml
. (#1303)
mamba
andconda
useenvironment.yml
if it existsvirtualenv
now requirespackaging
due todistutils
deprecations (#1240)- Wheels are now built for CPython
3.8, 3.9, 3.10, 3.11
- Packaging
requirements-dev.txt
file, used insetup.py
. (#1013)
- Adding environment variables to build and benchmark commands. (#809, #833)
- Added
--strict
option toasv run
to set exit code on failure. (#865) - Added
--no-stats
option toasv compare
andasv continuous
. (#879) - Added
--durations
option toasv run
andasv show
for displaying benchmark run durations. (#838) - Added
--date-period
option toasv run
for running benchmarks for commits separated by a constant time interval. (#835) - Web UI button to group regressions by benchmark. (#869)
- Space-saving v2 file format for storing results. (#847)
timeraw_*
benchmarks for measuring e.g. import times. (#832)- Support for using conda environment files for env setup. (#793)
- Results file format change requires
asv update
to update old data to v2 format. - The configuration syntax for "matrix", "exclude", and "include" in
asv.conf.json
has changed. The old syntax is still supported, unless you are installing packages namedreq
,env
,env_nobuild
.
- When an
asv find
step fails due to timeout, assume runtime equal to timeout to allow bisection to proceed (#768) - Minor fixes and improvements (#897, #896, #888, #881, #877, #876, #875, #861, #870, #868, #867, #866, #864, #863, #857, #786, #854, #855, #852, #850, #844, #843, #842, #839, #841, #840, #837, #836, #834, #831, #830, #829, #828, #826, #825, #824)
- Uniqueness of
repr()
forparam
objects is now guaranteed by suffixing unique identifier corresponding to order of appearance. (#771) - Memory addresses are now stripped from the
repr()
ofparam
elements, allowing comparison across multiple runs. (#771) asv dev
is now equivalent toasv run
with--python=same
default. (#874)asv continuous
by default now records measurement samples, for better comparison statistics. (#878)- ASV now uses PEP 518
pyproject.toml
in packaging. (#853)
- Change wheel installation default command to chdir away from build directory
instead of
--force-install
. (#823)
asv check
command for a quick check of benchmark suite validity. (#782)asv run HASHFILE:filename
can read commit hashes to run from file or stdin (#768)--set-commit-hash
option toasv run
, which allows recording results from runs in "existing" environments not managed by asv (#794)--cpu-affinity
option toasv run
and others, to set CPU affinity (#769)- "Hide legend" option in web UI (#807)
pretty_source
benchmark attribute for customizing source code shown (#810)- Record number of cores in machine information (#761)
- Default timer changed from
process_time()
totimeit.default_timer()
to fix resolution issues on Windows. The old behavior can be restored by settingBenchmark.timer = time.process_time
(#780)
- Fix pip command line in
install_command
(#806) - Python 3.8 compatibility (#814)
- Minor fixes and improvements (#759, #764, #767, #772, #779, #783, #784, #787, #790, #795, #799, #804, #812, #813, #815, #816, #817, #818, #820)
- In case of significant changes
asv continuous
message now reports if performance decreased or increased.
Minor bugfixes and improvements.
- Use measured uncertainties to weigh step detection. (#753)
- Detect also single-commit regressions, if significant. (#745)
- Use proper two-sample test when raw results available. (#754)
- Use a better regression "badness" measure. (#744)
- Display verbose command output immediately, not when command completes. (#747)
- Fix handling of benchmark suite import failures in forkserver and benchmark discovery. (#743, #742)
- Fix forkserver child process handling.
- In asv test suite, use dummy conda packages. (#738)
- Other minor fixes (#756, #750, #749, #746)
Major release with several new features.
- Revised timing benchmarking.
asv
will display and record the median and interquartile ranges of timing measurement results. The information is also used byasv compare
andasv continuous
in determining what changes are significant. Theasv run
command has new options for collecting samples. Timing benchmarks have new benchmarking parameters for controlling how timing works, includingprocesses
attribute for collect data by running benchmarks in different sequential processes. The defaults are adjusted to obtain faster benchmarking. (#707, #698, #695, #689, #683, #665, #652, #575, #503, #493) - Interleaved benchmark running. Timing benchmarks can be run in interleaved
order via
asv run --interleave-processes
, to obtain better sampling over long-time background performance variations. (#697, #694, #647) - Customization of build/install/uninstall commands. (#699)
- Launching benchmarks via a fork server (on Unix-based systems). Reduces the import time overheads in launching new benchmarks. Default on Linux. (#666, #709, #730)
- Benchmark versioning. Invalidate old benchmark results when benchmarks change,
via a benchmark
version
attribute. User-configurable, by default based on source code. (#509) - Setting benchmark attributes on command line, via
--attribute
. (#647) asv show
command for displaying results on command line. (#711)- Support for Conda channels. (#539)
- Provide ASV-specific environment variables to launched commands. (#624)
- Show branch/tag names in addition to commit hashes. (#705)
- Support for projects in repository subdirectories. (#611)
- Way to run specific parametrized benchmarks. (#593)
- Group benchmarks in the web benchmark grid (#557)
- Make the web interface URL addresses more copypasteable. (#608, #605, #580)
- Allow customizing benchmark display names (#484)
- Don't reinstall project if it is already installed (#708)
- The
goal_time
attribute in timing benchmarks is removed (and now ignored). See documentation on how to tune timing benchmarks now. asv publish
may ask you to runasv update
once after upgrading, to regeneratebenchmarks.json
ifasv run
was not yet run.- If you are using
asv
plugins, check their compatibility. The internal APIs inasv
are not guaranteed to be backward compatible.
- Fixes in 0.2.1 and 0.2.2 are also included in 0.3.
- Make
asv compare
accept named commits (#704) - Fix
asv profile --python=same
(#702) - Make
asv compare
behave correctly with multiple machines/envs (#687) - Avoid making too long result file names (#675)
- Fix saving profile data (#680)
- Ignore missing branches during benchmark discovery (#674)
- Perform benchmark discovery only when necessary (#568)
- Fix benchmark skipping to operate on a per-environment basis (#603)
- Allow putting
asv.conf.json
to benchmark suite directory (#717) - Miscellaneous minor fixes (#735, #734, #733, #729, #728, #727, #726, #723, #721, #719, #718, #716, #715, #714, #713, #706, #701, #691, #688, #684, #682, #660, #634, #615, #600, #573, #556)
- www: display regressions separately, one per commit (#720)
- Internal changes. (#712, #700, #681, #663, #662, #637, #613, #606, #572)
- CI/etc changes. (#585, #570)
- Added internal debugging command
asv.benchmarks
(#685) - Make tests not require network connection, except with Conda (#696)
- Drop support for end-of-lifed Python versions 2.6 & 3.2 & 3.3 (#548)
Prerelease. Same as 0.3rc1, minus #721--
Bugfix release with minor feature additions.
- Add a
--no-pull
option toasv publish
andasv run
(#592) - Add a
--rewrite
option toasv gh-pages
and fix bugs (#578, #529) - Add a
--html-dir
option toasv publish
(#545) - Add a
--yes
option toasv machine
(#540) - Enable running via
python -masv
(#538)
- Fix support for mercurial >= 4.5 (#643)
- Fix detection of git subrepositories (#642)
- Find conda executable in the "official" way (#646)
- Hide tracebacks in testing functions (#601)
- Launch virtualenv in a more sensible way (#555)
- Disable user site directory also when using conda (#553)
- Set PIP_USER to false when running an executable (#524)
- Set PATH for commands launched inside environments (#541)
- os.environ can only contain bytes on Win/py2 (#528)
- Fix hglib encoding issues on Python 3 (#508)
- Set GIT_CEILING_DIRECTORIES for Git (#636)
- Run pip via python -mpip to avoid shebang limits (#569)
- Always use https URLs (#583)
- Add a min-height on graphs to avoid a flot traceback (#596)
- Escape label html text in plot legends (#614)
- Disable pip build isolation in wheel_cache (#670)
- Fixup CI, test, etc issues (#616, #552, #601, #586, #554, #549, #571, #527, #560, #565)
Same as 0.2.2, minus #670.
- Use process groups on Windows (#489)
- Sanitize html filenames (#498)
- Fix incorrect date formatting + default sort order in web ui (#504)
- Automatic detection and listing of performance regressions. (#236)
- Support for Windows. (#282)
- New
setup_cache
method. (#277) - Exclude/include rules in configuration matrix. (#329)
- Command-line option for selecting environments. (#352)
- Possibility to include packages via pip in conda environments. (#373)
- The
pretty_name
attribute can be used to change the display name of benchmarks. (#425) - Git submodules are supported. (#426)
- The time when benchmarks were run is tracked. (#428)
- New summary web page showing a list of benchmarks. (#437)
- Atom feed for regressions. (#447)
- PyPy support. (#452)
- The parent directory of the benchmark suite is no longer inserted into
sys.path
. (#307) - Repository mirrors are no longer created for local repositories. (#314)
- In asv.conf.json matrix,
null
previously meant (undocumented) the latest version. Now it means that the package is to not be installed. (#329) - Previously, the
setup
andteardown
methods were run only once even when the benchmark method was run multiple times, for example due torepeat > 1
being present in timing benchmarks. This is now changed so that also they are run multiple times. (#316) - The default branch for Mercurial is now
default
, nottip
. (#394) - Benchmark results are now by default ordered by commit, not by date. (#429)
- When
asv run
and other commands are called without specifying revisions, the default values are taken from the branches inasv.conf.json
. (#430) - The default value for
--factor
inasv continuous
andasv compare
was changed from 2.0 to 1.1 (#469).
- Output will display on non-Unicode consoles. (#313, #318, #336)
- Longer default install timeout. (#342)
- Many other bugfixes and minor improvements.
Same as 0.2.
First full release.
- Display version correctly in docs.
- Include pip_requirements.txt.
No significant changes.
No significant changes.