7.2. What’s next in DataLad?

The datalad-next extension is a noteworthy extension not only because of the additional functionality it provides, but also because of its role in the development process of the datalad core package. It can be thought of as a staging area for additional functionality, or for improved performance and user experience. New features are introduced into datalad-next first, and, once tested for a sufficient amount of time and deemed useful, adopted in the core datalad package. As such, datalad-next isn’t confined to a topical set of features, but provides functionality with broad applicability. Some of this functionality takes the form of new stand-alone commands, while other functionality concerns improvements of existing commands under the hood via a special patch mechanism or modern re-implementations of established concepts.

As development in this extension is dynamic, the best place to find and up-to-date overview of functionality is in its source repository.

../_images/future_view.svg

7.2.1. Selected features

The ideal way to stay up-to-date on gooey developments is its documentation at docs.datalad.org/projects/next. A few selected commands are featured here, nevertheless.

7.2.1.1. datalad tree

The datalad tree command visualizes directory and dataset hierarchies. It mimics the UNIX/MS-DOS ‘tree’ utility to generate and display a directory tree, with DataLad-specific enhancements. With those, it can be used as an improved tree-style visualization, as a dataset discovery tool (when used with its --recursive option), or for programmatic directory traversal.

/tmp/mydir
 ├── [DS~0] ds_A/
    └── [DS~1] subds_A/
 └── [DS~0] ds_B/
     ├── dir_B/
        ├── file.txt
        ├── subdir_B/
        └── [DS~1] subds_B0/
     └── [DS~1] (not installed) subds_B1/

 5 datasets, 2 directories, 1 file

7.2.1.2. datalad download

The datalad download command is similar to datalad download-url, but more powerful and meant as a backend rather than a front-end. Unlike most DataLad commands, it can be used outside of the context of a dataset, has more options for input and output specifications (such as streaming), and provides support for additional URL schemes (such as SSH URLs).

7.2.1.3. datalad create-sibling-webdav

The datalad create-sibling-webdav extends DataLad’s create-sibling-* family with support for WebDAV services such as Nextcloud, owncloud Sciebo. Notably, it provides new, powerful dataset deposition methods that include human-readable worktree-like representations of datasets that are nevertheless clone-able.

7.2.2. Patches to exiting functionality

datalad-next differs from other extensions also in that it has a mechanism that hooks into existing DataLad commands and improves them under the hood. If the DataLad configuration extensions.load=next is set and datalad-next is installed, patches can be applied to select commands of the core package.

datalad configuration --scope global set datalad.extensions.load=next