btrfs package

Submodules

btrfs.crc32c module

btrfs.ctree module

This module contains Python object representations for btrfs metadata items.

Additionally, the helper FileSystem provides a convenient way to start exploring an online btrfs filesystem.

btrfs.ctree.U8(n)

Parameters:

n (int) – Any number.

Returns:

Unsigned 8 bit number.

Return type:

int

Example:

>>> btrfs.ctree.U8(64)
64
>>> btrfs.ctree.U8(-1)
255
>>> btrfs.ctree.U8(0x4000)
0

btrfs.ctree.ULL(n)

Parameters:

n (int) – Any number.

Returns:

Unsigned 64 bit number.

Return type:

int

Example:

>>> btrfs.ctree.ULL(64)
64
>>> btrfs.ctree.ULL(-1)
18446744073709551615
>>> btrfs.ctree.ULL(0x4000)
16384

btrfs.ctree.ROOT_TREE_OBJECTID = 1

Root tree

btrfs.ctree.EXTENT_TREE_OBJECTID = 2

Extent tree

btrfs.ctree.CHUNK_TREE_OBJECTID = 3

Chunk tree

btrfs.ctree.DEV_TREE_OBJECTID = 4

Device tree

btrfs.ctree.FS_TREE_OBJECTID = 5

Top level subvolume tree

btrfs.ctree.ROOT_TREE_DIR_OBJECTID = 6

Used in the root tree to store default subvolume information.

btrfs.ctree.CSUM_TREE_OBJECTID = 7

Checksum tree

btrfs.ctree.QUOTA_TREE_OBJECTID = 8

Quota tree

btrfs.ctree.UUID_TREE_OBJECTID = 9

Subvolume UUID tree

btrfs.ctree.FREE_SPACE_TREE_OBJECTID = 10

Free space tree

btrfs.ctree.DEV_STATS_OBJECTID = 0

Object ID of device statistics in the Device tree.

btrfs.ctree.BALANCE_OBJECTID = 18446744073709551612

Object ID to store balance status. (-4)

btrfs.ctree.ORPHAN_OBJECTID = 18446744073709551611

Object ID to store orphans that need cleaning. (-5)

btrfs.ctree.EXTENT_CSUM_OBJECTID = 18446744073709551606

Object ID used for checksum items. (-10)

btrfs.ctree.FREE_SPACE_OBJECTID = 18446744073709551605

Object ID for free space cache v1 items. (-11)

btrfs.ctree.FIRST_FREE_OBJECTID = 256

First available Object ID for subvolume trees.

btrfs.ctree.LAST_FREE_OBJECTID = 18446744073709551360

Last available Object ID for subvolume trees. (-256)

btrfs.ctree.FIRST_CHUNK_TREE_OBJECTID = 256

Object ID for Chunk objects in the Chunk tree.

btrfs.ctree.DEV_ITEMS_OBJECTID = 1

Object ID for Device items in the Device tree.

btrfs.ctree.INODE_ITEM_KEY = 1

Key type used by InodeItem

btrfs.ctree.INODE_REF_KEY = 12

Key type used by InodeRefList

btrfs.ctree.XATTR_ITEM_KEY = 24

Key type used by XAttrItemList

btrfs.ctree.ORPHAN_ITEM_KEY = 48

Key type used to track orphaned roots.

btrfs.ctree.DIR_ITEM_KEY = 84

Key type used by DirItemList

btrfs.ctree.DIR_INDEX_KEY = 96

Key type used by DirIndex

btrfs.ctree.EXTENT_DATA_KEY = 108

Key type used by FileExtentItem

btrfs.ctree.EXTENT_CSUM_KEY = 128

Key type used for checksum items.

btrfs.ctree.ROOT_ITEM_KEY = 132

Key type used by RootItem

btrfs.ctree.ROOT_REF_KEY = 156

Key type used by RootRef

btrfs.ctree.EXTENT_ITEM_KEY = 168

Key type used by ExtentItem

btrfs.ctree.METADATA_ITEM_KEY = 169

Key type used by MetaDataItem

btrfs.ctree.TREE_BLOCK_REF_KEY = 176

Key type used by TreeBlockRef

btrfs.ctree.EXTENT_DATA_REF_KEY = 178

Key type used by ExtentDataRef

btrfs.ctree.SHARED_BLOCK_REF_KEY = 182

Key type used by SharedBlockRef

btrfs.ctree.SHARED_DATA_REF_KEY = 184

Key type used by SharedDataRef

btrfs.ctree.BLOCK_GROUP_ITEM_KEY = 192

Key type used by BlockGroupItem

btrfs.ctree.FREE_SPACE_INFO_KEY = 198

Key type used by FreeSpaceInfo

btrfs.ctree.FREE_SPACE_EXTENT_KEY = 199

Key type used by FreeSpaceExtent

btrfs.ctree.FREE_SPACE_BITMAP_KEY = 200

Key type used by FreeSpaceBitmap

btrfs.ctree.DEV_EXTENT_KEY = 204

Key type used by DevExtent

btrfs.ctree.DEV_ITEM_KEY = 216

Key type used by DevItem

btrfs.ctree.CHUNK_ITEM_KEY = 228

Key type used by Chunk

btrfs.ctree.BALANCE_ITEM_KEY = 248

Balance status item key. Replaced by TEMPORARY_ITEM_KEY.

btrfs.ctree.TEMPORARY_ITEM_KEY = 248

Key for various short term persistent stored items.

btrfs.ctree.DEV_STATS_KEY = 249

Device statistics key. Replaced by PERSISTENT_ITEM_KEY.

btrfs.ctree.PERSISTENT_ITEM_KEY = 249

Key for various long term persistent stored items.

btrfs.ctree.BLOCK_GROUP_SINGLE = 0

Block Group single type. Does not exist in kernel code.

btrfs.ctree.BLOCK_GROUP_DATA = 1

Block Group DATA type.

btrfs.ctree.BLOCK_GROUP_SYSTEM = 2

Block Group SYSTEM type.

btrfs.ctree.BLOCK_GROUP_METADATA = 4

Block Group METADATA type.

btrfs.ctree.BLOCK_GROUP_RAID0 = 8

Block Group RAID0 profile.

btrfs.ctree.BLOCK_GROUP_RAID1 = 16

Block Group RAID1 profile.

btrfs.ctree.BLOCK_GROUP_DUP = 32

Block Group DUP profile.

btrfs.ctree.BLOCK_GROUP_RAID10 = 64

Block Group RAID10 profile.

btrfs.ctree.BLOCK_GROUP_RAID5 = 128

Block Group RAID5 profile.

btrfs.ctree.BLOCK_GROUP_RAID6 = 256

Block Group RAID6 profile.

btrfs.ctree.BLOCK_GROUP_RAID1C3 = 512

Block Group RAID1C3 profile.

btrfs.ctree.BLOCK_GROUP_RAID1C4 = 1024

Block Group RAID1C4 profile.

btrfs.ctree.BLOCK_GROUP_TYPE_MASK = 7

All Block Group type bits (data, system, metadata).

btrfs.ctree.BLOCK_GROUP_PROFILE_MASK = 2040

All Block Group profile bits (raid1, dup, etc…).

btrfs.ctree.qgroup_level(objectid)

Helper to get qgroup level from a qgroup relation objectid.

Parameters:

objectid (int) – 64-bit object ID field.

Returns:

qgroup level.

Return type:

int

btrfs.ctree.qgroup_subvid(objectid)

Helper to get qgroup subvolume ID from a qgroup relation objectid.

Parameters:

objectid (int) – 64-bit object ID field.

Returns:

qgroup subvolume ID.

Return type:

int

exception btrfs.ctree.ItemNotFoundError

Bases: IndexError

Helper exception for lookup convenience functions.

If a convenience function on a btrfs.ctree.FileSystem object is supposed to return exactly one object at a specific location, and no object is found, this type of exception is raised.

An example is the block_group() helper, which raises this error if no block group item is found at the exact specified location.

class btrfs.ctree.Key(objectid, type_, offset)

Bases: object

Btrfs metadata trees have a key space of 136-bit numbers.

A full 136-bit tree key is composed as:

(objectid << 72) + (type << 64) + offset

Parameters:

• objectid (Union[int, str]) – 64-bit object ID number or string representation.

• type_ (Union[int, str]) – 8-bit type number or string representation.

• offset (Union[int, str]) – 64-bit offset number or string representation.

Key objects support sorting and simple addition and subtraction. Also, when subtracting 1 from a zero key, the value wraps around to the largest value possible, vice versa.

Example:

>>> key1 = btrfs.ctree.Key(425, 'DIR_ITEM', 17818406)
>>> key1
Key(425, 84, 17818406)
>>> str(key1)
'(425 DIR_ITEM 17818406)'
>>> key2 = btrfs.ctree.Key(442, btrfs.ctree.EXTENT_DATA_KEY, 0)
>>> key2 > key1
True

>>> min_key = btrfs.ctree.Key(0, 0, 0)
>>> min_key
Key(0, 0, 0)
>>> str(min_key)
'(0 0 0)'
>>> min_key - 1
Key(18446744073709551615, 255, 18446744073709551615)
>>> str(min_key - 1)
'(-1 255 -1)'

The -1 value in the string representation is just a convenience way to write the maximum allowed number. The actual value for a 64 bit numer is still 18446744073709551615, and for 8 bit that’s 255 of course.

For example, when setting up a minimum and maximum key for a metadata search, the arithmetic that can be done helps quickly defining the maximum value. The next example shows the key range for finding all intormation about an inode in a filesystem tree:

Example:

>>> inum = 31337
>>> min_key = btrfs.ctree.Key(inum, 0, 0)
>>> max_key = btrfs.ctree.Key(inum + 1, 0, 0) - 1
>>>
>>> min_key
Key(31337, 0, 0)
>>> max_key
Key(31337, 255, 18446744073709551615)

Last but not least, the utils module contains the helper function parse_key_string() to dissect a full text key string:

Example:

>>> btrfs.utils.parse_key_string('(535 EXTENT_DATA 0)')
Key(535, 108, 0)

property objectid

Key Object ID

property type

Key Type

property offset

Key Offset

property key

Full numeric 136-bit key value.

class btrfs.ctree.DiskKey(data)

Bases: Key

Object representation of struct btrfs_disk_key.

Objects of this type are used in metadata search results.

class btrfs.ctree.FileSystem(path)

Bases: object

The FileSystem object is a bit of a spider in the web of this library. It contains a lot of convenience methods providing quick access to all kinds of functionality.

Parameters:

path (str) – Path to the mounted filesystem.

Variables:

• path (str) – The filesystem path used to initialize this object.

• fsid (uuid.UUID) – Filesystem ID.

• nodesize (int) – B-tree node size (same as leaf size).

• sectorsize (int) – Smallest allocatable block size in bytes for storing data.

The fsid, nodesize and sectorsize values are cached from a call to fs_info() when initializing the object.

It is highly recommended to use the built in context manager. Doing so prevents leaking the internal open file descriptor.

Example:

>>> with btrfs.ctree.FileSystem('/') as fs:
...     print(fs.top_level().generation)
...
3382004

fs_info()

Returns:

General filesystem information.

Return type:

btrfs.ioctl.FsInfo

dev_info(devid)

Parameters:

devid (int) – Device ID.

Returns:

Device information.

Return type:

btrfs.ioctl.DevInfo

dev_stats(devid, reset=False)

Parameters:

• devid (int) – Device ID.

• reset (bool) – Reset device error counters to zero.

Returns:

Device statistics.

Return type:

btrfs.ioctl.DevStats

space_info()

Returns:

Space information

Return type:

List[btrfs.ioctl.SpaceInfo]

search(tree, min_key=None, max_key=None)

Retrieve all metadata items within a specific range. This is basically a thin wrapper around the search_v2() with a bit limited functionality, but suited for almost all use cases when quickly searching around.

Parameters:

• tree (int) – The metadata tree we’re searching in.

• min_key (btrfs.ctree.Key) – Minimum key value for items to return.

• max_key (btrfs.ctree.Key) – Maximum key value for items to return.

Returns:

Any metadata item found in the search range, as sub class of ItemData, helped by the btrfs.ctree.classify() function.

Return type:

Iterator[ItemData]

devices(min_devid=1, max_devid=18446744073709551615)

Parameters:

• min_devid (int) – Lowest Device ID to search for.

• max_devid (int) – Highest Device ID to search for.

Returns:

Device Items from the Chunk tree.

Return type:

Iterator[DevItem]

chunks(min_vaddr=0, max_vaddr=18446744073709551615, nr_items=None)

Parameters:

• min_vaddr (int) – Lowest virtual address to search for.

• max_vaddr (int) – Highest virtual address to search for.

• nr_items (int) – Maximum amount of items to return. Defaults to no limit.

Returns:

Chunk items from the Chunk tree.

Return type:

Iterator[Chunk]

dev_extents(min_devid=1, max_devid=18446744073709551615)

Parameters:

• min_devid (int) – Lowest Device ID to search for.

• max_devid (int) – Highest Device ID to search for.

Returns:

Device Extent Items from the Device tree.

Return type:

Iterator[DevExtent]

block_group(vaddr, length=None)

Parameters:

• vaddr (int) – Starting virtual address of the block group.

• length (int) – Block group length (optional). If this information is already known, it can be used to construct an exact match for the search key.

Returns:

Block Group Item

Return type:

BlockGroupItem

Raises:

ItemNotFoundError if no Block Group Item can be found at the address.

extents(min_vaddr=0, max_vaddr=18446744073709551615, load_data_refs=False, load_metadata_refs=False)

Parameters:

• min_vaddr (int) – Lowest virtual address to search for.

• max_vaddr (int) – Highest virtual address to search for.

• load_data_refs (bool) – Parse and load backreference information for data extents.

• load_metadata_refs (bool) – Parse and load backreference information for metadata extents.

Returns:

Extent and MetaData Items from the Extent tree

Return type:

Iterator[Union[ExtentItem, MetaDataItem]]

The ‘refs’ are backreference information. These sub items are stored inside the ExtentItem and MetaDataItem Items, and overflow to separately indexed items. When dealing with search results in user space, these backreferences are of little use to us, since the search API only allows us to search in tree leaves. So, they’re ignored by default.

top_level()

Returns:

The top level subvolume with ID 5, a.k.a. FS_TREE_OBJECTID.

Return type:

RootItem

subvolumes(min_id=256, max_id=18446744073709551360)

Parameters:

• min_id (int) – Lowest subvolume ID to search for.

• max_id (int) – Highest subvolume ID to search for.

Returns:

Root Items from the Root tree, containing subvolume information.

Return type:

Iterator[RootItem]

orphan_subvol_ids()

Returns:

ObjectID numbers of orphaned items in the Root tree.

Return type:

List[int]

free_space_extents(min_vaddr=0, max_vaddr=18446744073709551615)

Parameters:

• min_vaddr (int) – Minimum virtual address when searching for free space.

• max_vaddr (int) – Maximum virtual address when searching for free space.

Returns:

Free space extent information from the Free Space Tree.

Return type:

Iterator[btrfs.free_space_tree.FreeSpaceExtent]

Note

The Free Space Tree can contain both Free Space Extent Items and Free Space Bitmap Items, which contain a more compact representation of free space extents. This helper function will transparently unpack these bitmaps and return one type of helper object, the FreeSpaceExtent object.

sync()

Call the btrfs sync kernel function, causing a transaction commit.

features()

Returns:

Filesystem Features.

Return type:

btrfs.ioctl.FeatureFlags

mixed_groups()

Returns:

True if this filesystem used mixed block groups with metadata and data in the same block groups, else False.

Return type:

bool

usage()

Returns:

Detailed filesystem usage information.

Return type:

btrfs.fs_usage.FsUsage

class btrfs.ctree.ItemData(header)

Bases: object

ItemData is a base class for all tree item types.

Variables:

key – Key under which this item is stored in the tree.

class btrfs.ctree.SubItem

Bases: object

class btrfs.ctree.DevItem(header, data)

Bases: ItemData

Object representation of struct btrfs_dev_item.

A DevItem contains information about a single block device that is attached to the filesystem.

• Tree: CHUNK_TREE_OBJECTID (3)

• Key objectid: DEV_ITEMS_OBJECTID (1)

• Key type: DEV_ITEM_KEY (216)

• Key offset: Device ID.

Variables:

• devid (int) – Device ID.

• total_bytes (int) – Total amount of bytes.

• bytes_used (int) – Total amount of allocated bytes.

• io_align (int) – Not used, set to same value as sector_size.

• io_width (int) – Not used, set to same value as sector_size.

• sector_size (int) – Smallest IO block size to use.

• type (int) – Not used

• generation (int) – Not used

• start_offset (int) – Not used

• dev_group (int) – Not used

• seek_speed (int) – Not used

• bandwidth (int) – Not used

• uuid (uuid.UUID) – Device UUID.

• fsid (uuid.UUID) – Filesystem ID.

class btrfs.ctree.Chunk(header, data)

Bases: ItemData

Object representation of struct btrfs_chunk.

A Chunk is a piece of virtual address space. A Chunk has a 1 to 1 relationship to a BlockGroupItem, and a 1 to many relationship with a fixed amount of Stripe objects.

• Tree: CHUNK_TREE_OBJECTID (3)

• Key objectid: FIRST_CHUNK_TREE_OBJECTID (256)

• Key type: CHUNK_ITEM_KEY (228)

• Key offset: Virtual address.

Variables:

• vaddr (int) – Virtual address where the Chunk starts (taken from the offset field of the item key).

• length (int) – Chunk length in bytes,

• owner (int) – Extent tree the chunk belongs to. Not used, always 2 now.

• stripe_len (int) – Hardcoded to BTRFS_STRIPE_LEN, which is 64kiB.

• type (int) – Block group flags for the corresponding block group. So, a Chunk type contains both Block Group type and profile.

• io_align (int) – Not used, see stripe_len.

• io_width (int) – Not used, see stripe_len.

• sector_size (int) – Smallest IO block size to use.

• num_stripes (int) – Amount of Stripe (or, also the amount of Device Extent) objects related to this Chunk.

• sub_stripes (int) – A hack for RAID10. For RAID10 this value is 2, otherwise 1.

• stripes (List[Stripe]) – Stripe Items that are stored inside this Chunk Item.

class btrfs.ctree.Stripe(data)

Bases: SubItem

Object representation of struct btrfs_stripe.

A list of Stripe items is hidden inside the Chunk item and each of them contains information that points to the beginning of a Device Extent, which is an actual allocated piece of physical disk space in which data ends up that is written to the virtual address space of the Chunk.

Variables:

• devid (int) – Device ID of the device that holds the Device Extent.

• offset (int) – Physical address on the device where the Device Extent starts.

• uuid (uuid.UUID) – Device UUID of the device with the above listed devid.

class btrfs.ctree.DevExtent(header, data)

Bases: ItemData

Object representation of struct btrfs_dev_extent.

The Device Extent is a range of physical address space allocated from one of the attached devices and used by a Chunk to store data.

• Tree: DEV_TREE_OBJECTID (4)

• Key objectid: Device ID.

• Key type: DEV_EXTENT_KEY (204)

• Key offset: Physical address.

Variables:

• devid (int) – Device ID of the device that holds this Device Extent (taken from the objectid field of the item key).

• paddr (int) – Physical address on the device where the Device Extent starts (taken from the offset field of the item key).

• chunk_tree (int) – Chunk tree the device extent belongs to. This is always 3 now.

• chunk_offset (int) – Virtual address of the related Chunk.

• length (int) – Length in physical bytes.

• chunk_tree_uuid (uuid.UUID) – UUID of the chunk tree that this Device Extent belongs to. This is currently always the UUID of tree 3.

property vaddr

Alias for the chunk_offset attribute.

class btrfs.ctree.BlockGroupItem(header, data)

Bases: ItemData

Object representation of struct btrfs_block_group_item.

The Block Group has a 1 to 1 relationship with a Chunk and tracks some usage information about a range of virtual address space.

• Tree: EXTENT_TREE_OBJECTID (2)

• Key objectid: Virtual address.

• Key type: BLOCK_GROUP_ITEM_KEY (192)

• Key offset: Block Group length.

Variables:

• vaddr (int) – Virtual address where the Bock Group starts (taken from the objectid field of the item key).

• length (int) – Block Group length in bytes (taken from the offset field of the item key).

• used (int) – Amount of bytes used by Extents in the Block Group.

• chunk_objectid (int) – Object ID of the Chunk this Block Group relates to. Currently always 256.

• flags (int) – Type and profile for this Block Group. e.g. 0x11, which is DATA|RAID1.

property used_pct

Convenience property that calculates the percentage of usage.

class btrfs.ctree.ExtentItem(header, data, load_data_refs=True, load_metadata_refs=True)

Bases: ItemData

Object representation of struct btrfs_extent_item.

An ExtentItem lives in the Extent Tree and tracks information about a piece of virtual address space that is in use. A FileExtentItem object from a subvolume tree can point to it, to let us know a file in the filesystem uses part of this extent.

• Tree: EXTENT_TREE_OBJECTID (2)

• Key objectid: Virtual address.

• Key type: EXTENT_ITEM_KEY (168)

• Key offset: Extent length.

If the skinny_metadata feature is enabled in the filesystem, then metadata extents are stored separately as MetaDataItem.

An ExtentItem includes inline backreference items. In the kernel, this information is used to be able to find out which different inodes in subvolume trees are using data from this extent. For us, in userspace, this information is not very relevant, since we cannot look into B-tree nodes using the kernel search API. So, by default the backreference information is ignored when creating these kind of objects from a metadata search.

Instead, to find out who is referencing data from an extent, the btrfs.ioctl.logical_to_ino() and btrfs.ioctl.logical_to_ino_v2() functions can be used.

Variables:

• vaddr (int) – Virtual address where the Extent starts (taken from the objectid field of the item key).

• length (int) – Length of the extent in bytes (taken from the offset field of the item key).

• refs (int) – Amount of explicit references to this extent.

• generation (int) – Generation of the filesystem when this extent was created.

• flags (int) – Some flags describing which type of extent this is.

The flags are an or-ed combination of one or more of the following values (available as attribute of this module):

• EXTENT_FLAG_DATA: This extent contains data.

• EXTENT_FLAG_TREE_BLOCK: This extent contains a metadata tree block.

• BLOCK_FLAG_FULL_BACKREF: The tree block backreference contains a full back reference.

When backreference information is being loaded, there are a few additional lists present in this object. Also, when using the FileSystem.extents() helper to retrieve extent information, then separately stored backreference items which do not fit into the extent item itself any more are also appended to these lists:

If the extent is a data extent, then this object contains:

Variables:

• extent_data_refs (List[Union[InlineExtentDataRef, ExtentDataRef]]) – Indirect back references.

• shared_data_refs (List[Union[InlineSharedDataRef, SharedDataRef]]) – Shared back references.

If the extent is a metadata tree block, then this object contains:

Variables:

• tree_block_refs (List[Union[InlineTreeBlockRef, TreeBlockRef]]) – Tree block backreferences.

• shared_block_refs (List[Union[InlineSharedBlockRef, SharedBlockRef]]) – Shared tree block backreferences.

Further documentation of backreferences is out of scope for this module. Please refer to the btrfs wiki about resolving extent backreferences for more information.

class btrfs.ctree.ExtentDataRef(header, data)

Bases: ItemData

Object representation of struct btrfs_extent_data_ref.

Documentation of this item is out of scope for this module. Please refer to the btrfs wiki about resolving extent backreferences for more information.

• Tree: EXTENT_TREE_OBJECTID (2)

• Key type: EXTENT_DATA_REF_KEY (178)

Variables:

• root (int) – root

• objectid (int) – objectid

• offset (int) – offset

• count (int) – count

class btrfs.ctree.InlineExtentDataRef(data)

Bases: ExtentDataRef

Identical content to ExtentDataRef, but the backreference was inlined in the extent item.

class btrfs.ctree.SharedDataRef(header, data)

Bases: ItemData

Object representation of struct btrfs_shared_data_ref.

Documentation of this item is out of scope for this module. Please refer to the btrfs wiki about resolving extent backreferences for more information.

• Tree: EXTENT_TREE_OBJECTID (2)

• Key type: SHARED_DATA_REF_KEY (184)

Variables:

• parent (int) – parent

• count (int) – count

class btrfs.ctree.InlineSharedDataRef(data)

Bases: SharedDataRef

Identical content to SharedDataRef, but the backreference was inlined in the extent item.

class btrfs.ctree.TreeBlockInfo(data)

Bases: SubItem

Object representation of struct btrfs_tree_block_info.

Documentation of this item is out of scope for this module. Please refer to the btrfs wiki about resolving extent backreferences for more information.

Variables:

• key (Key) – key

• level (int) – level

class btrfs.ctree.MetaDataItem(header, data, load_refs=True)

Bases: ItemData

Object representation of struct btrfs_metadata_item.

A MetaDataItem lives in the Extent Tree and tracks information about a piece of virtual address space that is in use to store a metadata tree block.

• Tree: EXTENT_TREE_OBJECTID (2)

• Key objectid: Virtual address.

• Key type: METADATA_ITEM_KEY (169)

• Key offset: Extent length.

If the skinny_metadata filesystem feature is enabled, metadata extents are tracked using this item type, which encodes all necessecary data in a more compact way than using a regular extent item.

Variables:

• vaddr (int) – Virtual address where the Extent starts (taken from the objectid field of the item key).

• skinny_level (int) – Tree level. field of the item key).

• refs (int) – Amount of explicit references to this extent.

• generation (int) – Generation of the filesystem when this extent was created.

• flags (int) – See below.

The flags are an or-ed combination of one or more of the following values (available as attribute of this module):

• EXTENT_FLAG_TREE_BLOCK: This extent contains a metadata tree block.

• BLOCK_FLAG_FULL_BACKREF: The tree block backreference contains a full back reference.

When backreference information is being loaded, there are a few additional lists present in this object. Also, when using the FileSystem.extents() helper to retrieve extent information, then separately stored backreference items which do not fit into the extent item itself any more are also appended to these lists:

Variables:

• tree_block_refs (List[Union[InlineTreeBlockRef, TreeBlockRef]]) – Tree block backreferences.

• shared_block_refs (List[Union[InlineSharedBlockRef, SharedBlockRef]]) – Shared tree block backreferences.

Further documentation of backreferences is out of scope for this module. Please refer to the btrfs wiki about resolving extent backreferences for more information.

class btrfs.ctree.TreeBlockRef(header)

Bases: ItemData

Tree block reference

Documentation of this item is out of scope for this module. Please refer to the btrfs wiki about resolving extent backreferences for more information.

• Tree: EXTENT_TREE_OBJECTID (2)

• Key type: TREE_BLOCK_REF_KEY (176)

Variables:

root (int) – root

class btrfs.ctree.InlineTreeBlockRef(root)

Bases: TreeBlockRef

Identical content to TreeBlockRef, but the backreference was inlined in the extent item.

class btrfs.ctree.SharedBlockRef(header)

Bases: ItemData

Shared tree block reference

Documentation of this item is out of scope for this module. Please refer to the btrfs wiki about resolving extent backreferences for more information.

• Tree: EXTENT_TREE_OBJECTID (2)

• Key type: SHARED_BLOCK_REF_KEY (182)

Variables:

parent (int) – parent

class btrfs.ctree.InlineSharedBlockRef(parent)

Bases: SharedBlockRef

Identical content to SharedBlockRef, but the backreference was inlined in the extent item.

class btrfs.ctree.TimeSpec(data)

Bases: object

Object representation of struct btrfs_timespec.

The TimeSpec type of item is used embedded in other metadata items when a time value needs to be stored. Examples are the mtime, ctime etc fields in an inode item.

It’s also possible to create objects of this type manually. To do so, use the static from_values() helper function. (The regular object constructor is reserved for the code parsing metadata items from search queries.)

Variables:

• sec (int) – seconds

• nsec (int) – nanoseconds

Example:

>>> my_time = btrfs.ctree.TimeSpec.from_values(1546280270, 4044945)
>>> my_time.sec
1546280270
>>> my_time.nsec
4044945
>>> my_time.iso8601
'2018-12-31T18:17:50.404495'

static from_values(sec, nsec)

Create a Timespec object

Parameters:

• sec (int) – seconds

• nsec (int) – nanoseconds

property iso8601

Return the timestamp as ISO8601 formatted string.

class btrfs.ctree.InodeItem(header, data)

Bases: ItemData

Object representation of struct btrfs_inode_item.

The inode item stores information of a single file or directory. Not the name, because a file can have multiple names.

• Tree: FS_TREE_OBJECTID (5) or any other subvolume tree.

• Key objectid: Inode number.

• Key type: INODE_ITEM_KEY (1)

• Key offset: 0

Variables:

• objectid (int) – The inode number. (taken from the objectid field of the item key).

• generation (int) – Generation of the filesystem when the inode was created.

• transid (int) – Generation of the filesystem when the inode was last changed.

• size (int) – File size in bytes.

• nbytes (int) – Allocated disk blocks for this file in bytes.

• block_group (int) – Only used for free space cache v1, for which it’s the related block group virtual address.

• nlink (int) – Amount of hardlinks the file has.

• uid (int) – Numerical user id of the owner of the file.

• gid (int) – Numerical group id of the owner of the file.

• mode (int) – File permissions.

• rdev (int) – Major and minor device numbers for special files.

• flags (int) – Inode flags, see below.

• sequence (int) – Sequence number for NFS.

• atime (TimeSpec) – Time of last access.

• ctime (TimeSpec) – Time of last file metadata change. Also updated when file contents change.

• mtime (TimeSpec) – Time of last modification to file contents.

• otime (TimeSpec) – Time of file birth.

The flags are an or-ed combination of one or more of the following values (available as attribute of this module):

• INODE_NODATASUM

• INODE_NODATACOW

• INODE_READONLY

• INODE_NOCOMPRESS

• INODE_PREALLOC

• INODE_SYNC

• INODE_IMMUTABLE

• INODE_APPEND

• INODE_NODUMP

• INODE_NOATIME

• INODE_DIRSYNC

• INODE_COMPRESS

class btrfs.ctree.InodeRefList(header, data)

Bases: ItemData, MutableSequence

A collection of struct btrfs_inode_ref indexed under a single tree key.

A InodeRefList is a list of InodeRef objects which contain information about every name the inode is known under in a single directory. So, when we already know the inode number of a file, we can find in which places it has hardlinks pointing at it.

• Tree: FS_TREE_OBJECTID (5) or any other subvolume tree.

• Key objectid: Inode number.

• Key type: INODE_REF_KEY (12)

• Key offset: Inode number of the containing directory.

This class is a helper that does not exist in Btrfs itself.

Besides acting as an immutable list of InodeRef objects, there are some additional attributes:

Variables:

• objectid (int) – Inode number of the file. (taken from the objectid field of the item key)

• parent_objectid (int) – Inode number of the containing directory. (taken from the offset field of the item key)

insert(index, value)

Not implemented.

class btrfs.ctree.InodeRef(data, pos)

Bases: SubItem

Object representation of struct btrfs_inode_ref.

Also see InodeRefList.

Variables:

• index (int) – Directory index number in the containing directory. Refer to the parent_objectid attribute of the InodeRefList object that contains this InodeRef to find the inode number of the directory.

• name_len (int) – Amount of bytes used to store the filename.

• name (bytes) – Filename as bytes.

class btrfs.ctree.InodeExtrefList(header, data)

Bases: ItemData, MutableSequence

A collection of struct btrfs_inode_extref indexed under a single tree key.

A InodeExtrefList is a list of InodeExtref objects. By default, names under which a file is known are stored in the InodeRefList of InodeRef. However, if a file has so many hardlinks that that item would become bigger than a metadata page, the rest of the items are stored separately.

• Tree: FS_TREE_OBJECTID (5) or any other subvolume tree.

• Key objectid: Inode number.

• Key type: INODE_EXTREF_KEY (13)

• Key offset: extref_hash() of the filename.

The InodeExtref item is a list because multiple different filenames can end up having the same crc32 value.

This class is a helper that does not exist in Btrfs itself.

Besides acting as an immutable list of InodeExtRef objects, there are some additional attributes:

Variables:

• objectid (int) – Inode number of the file. (taken from the objectid field of the item key)

• extref_hash (int) – extref_hash() of the filename. (taken from the offset field of the item key)

insert(index, value)

Not implemented.

class btrfs.ctree.InodeExtref(data, pos)

Bases: object

Object representation of struct btrfs_inode_extref.

Also see InodeExtrefList.

Variables:

• parent_objectid (int) – Inode number of the containing directory.

• index (int) – Directory index number in the containing directory.

• name_len (int) – Amount of bytes used to store the filename.

• name (bytes) – Filename as bytes.

class btrfs.ctree.DirItemList(header, data)

Bases: ItemData, MutableSequence

A collection of struct btrfs_dir_item indexed under a single tree key.

A DirItemList is a list of DirItem objects. Based on a filename hash, they point to the inode item for the corresponding file. Since multiple different filenames can end up having the same name hash, this item can contain multiple DirItem objects.

• Tree: FS_TREE_OBJECTID (5) or any other subvolume tree.

• Key objectid: Inode number of the directory.

• Key type: DIR_ITEM_KEY (84)

• Key offset: name_hash() of the filename.

This class is a helper that does not exist in Btrfs itself.

Besides acting as an immutable list of DirItem objects, there are some additional attributes:

Variables:

• objectid (int) – Inode number of the directory. (taken from the objectid field of the item key)

• name_hash (int) – name_hash() of the filename. (taken from the offset field of the item key)

insert(index, value)

Not implemented.

class btrfs.ctree.XAttrItemList(header, data)

Bases: DirItemList

A collection of struct btrfs_dir_item, used to store extended attributes and indexed under a single tree key.

An XAttrItemList is a list of XAttrItem objects, which contain key value pairs in the name and data fields of the dir_item struct. Since multiple different keys can end up having the same name hash, this item can contain multiple XAttrItem objects.

• Tree: FS_TREE_OBJECTID (5) or any other subvolume tree.

• Key objectid: Inode number of a file on which the xattr is set.

• Key type: XATTR_ITEM_KEY (24)

• Key offset: name_hash() of the key.

This class is a helper that does not exist in Btrfs itself.

Besides acting as an immutable list of XAttrItem objects, there are some additional attributes:

Variables:

• objectid (int) – Inode number of a file on which the xattr is set. (taken from the objectid field of the item key)

• name_hash (int) – name_hash() of the xattr key. (taken from the offset field of the item key)

class btrfs.ctree.DirItem(data, pos)

Bases: SubItem

Object representation of struct btrfs_dir_item.

Based on the name hash of a filename, this object directly points to the inode item for the corresponding file. Using this mapping allows quick file access when the name is known, even in directories with a large amount of files in them.

Also see DirItemList.

Variables:

• location (DiskKey) – Key of the file inode item.

• transid (int) – Generation of the filesystem when this item was created.

• data_len (int) – Not used, always 0.

• name_len (int) – Amount of bytes used to store the filename.

• type (int) – File type that is represented by the inode item that is being referenced.

• name (bytes) – Filename as bytes.

File type is one of the following values (available as attribute of this module):

• FT_UNKNOWN

• FT_REG_FILE

• FT_DIR

• FT_CHRDEV

• FT_BLKDEV

• FT_FIFO

• FT_SOCK

• FT_SYMLINK

• FT_XATTR

• FT_MAX

class btrfs.ctree.XAttrItem(data, pos)

Bases: DirItem

Object representation of struct btrfs_dir_item, used to store extended attribute information.

This object contains a key and value for an extended attribute on a file. It reuses the dir_item data structure.

Also see XattrItemList.

Variables:

• transid (int) – Generation of the filesystem when this item was created.

• name_len (int) – Amount of bytes used to store the key.

• data_len (int) – Amount of bytes used to store the value.

• name (bytes) – Key as bytes.

• data (bytes) – Value as bytes.

class btrfs.ctree.DirIndex(header, data)

Bases: ItemData

Object representation of struct btrfs_dir_item, but used to store filenames ordered on directory index.

The DirIndex objects list the contents of a directory in the order in which items were added.

• Tree: FS_TREE_OBJECTID (5) or any other subvolume tree.

• Key objectid: Inode number of the directory.

• Key type: DIR_INDEX_KEY (96)

• Key offset: Index in the directory.

Variables:

• objectid (int) – Inode number of the directory. (taken from the objectid field of the item key)

• index (int) – Index in the directory. (taken from the offset field of the item key)

• location (DiskKey) – Key of the file inode item.

• transid (int) – Generation of the filesystem when this item was created.

• data_len (int) – Not used, always 0.

• name_len (int) – Amount of bytes used to store the filename.

• type (int) – File type that is represented by the inode item that is being referenced.

• name (bytes) – Filename as bytes.

File type is one of the following values (available as attribute of this module):

• FT_UNKNOWN

• FT_REG_FILE

• FT_DIR

• FT_CHRDEV

• FT_BLKDEV

• FT_FIFO

• FT_SOCK

• FT_SYMLINK

• FT_XATTR

• FT_MAX

class btrfs.ctree.RootItem(header, data)

Bases: ItemData

Object representation of struct btrfs_root_item.

The RootItem lives in the root tree, a.k.a. the ‘tree of trees’. It contains information about the root metadata node of another tree.

• Tree: ROOT_TREE_OBJECTID (1).

• Key objectid: Tree ID.

• Key type: ROOT_ITEM_KEY (132)

• Key offset: Index in the directory.

Variables:

• objectid (int) – Tree ID. (taken from the objectid field of the item key)

• inode (InodeItem) – Embedded inode item. Only the flags field of it is used.

• generation (int) – Generation of the filesystem when this root was created.

• root_dirid (int) – Objectid for the root directory in a subvolume tree (always 256 in that case). 0 for other trees.

• bytenr (int) – Virtual address of the root node of this tree.

• byte_limit (int) – Not used

• bytes_used (int) – Not used

• last_snapshot (int) – The generation of the filesystem when the most recent snapshot of a subvolume was made.

• flags (int) – See below.

• refs (int) – Not used, either 0 or 1.

• drop_progress (DiskKey) – Key of the last removed item during cleanup of a removed subvolume.

• drop_level (int) – The tree level of the metadata node or leaf that contains the key from drop_progress.

• level (int) – The height of the tree that this root item refers to.

The flags are an or-ed combination of one or more of the following values (available as attribute of this module):

• ROOT_SUBVOL_RDONLY: The subvolume is read only.

The following fields were introduced in Linux 3.6. Btrfs would still allow using the filesystem with an older kernel, but if the content of generation_v2 does not match generation, all new fields would be invalidated:

Variables:

• generation_v2 (int) – Same value as generation.

• uuid (uuid.UUID) – Subvolume UUID.

• parent_uuid (uuid.UUID) – Subvolume UUID that this subvolume is a snapshot of.

• received_uuid (uuid.UUID) – Subvolume UUID of the subvolume that this subvolume was duplicated from using send/receive.

• ctransid (int) – Generation when the tree was last modified.

• otransid (int) – Generation when the tree was created.

• stransid (int) – Generation of the filesystem from which a subvolume was sent. Only used if this is a received subvolume.

• rtransid (int) – Generation of this filesystem when the subvolume was received.

• ctime (TimeSpec) – Timestamp for ctransid.

• otime (TimeSpec) – Timestamp for otransid.

• stime (TimeSpec) – Timestamp for stransid.

• rtime (TimeSpec) – Timestamp for rtransid.

class btrfs.ctree.RootRef(header, data)

Bases: ItemData

Object representation of btrfs_root_ref.

The RootRef item lives in the root tree, a.k.a. the ‘tree of trees’. It contains information about the place where a subvolume is located inside another subvolume.

• Tree: ROOT_TREE_OBJECTID (1).

• Key objectid: Parent tree ID.

• Key type: ROOT_REF_KEY (156)

• Key offset: Tree ID of the subvolume.

Variables:

• parent_tree (int) – Containing Tree ID. (taken from the objectid field of the item key)

• tree (int) – Tree ID of the subvolume. (taken from the offset field of the item key)

• dirid (int) – Inode number of the containing directory.

• sequence (int) – Directory index number in the containing directory.

• name_len (int) – Amount of bytes used to store the filename.

• name (bytes) – Filename as bytes.

When combining this information with a call to the ino_lookup ioctl, we can quickly figure out the relative path inside the containing subvolume.

E.g. for (259 ROOT_REF 1052) in the root tree, with dirid 20197, sequence 65 and name baz, we can do btrfs.ioctl.ino_lookup(fd, 259, 20197). The result is e.g. name_bytes=b’foo/bar/’, so the location inside the containing subvolume is foo/bar/baz.

When doing such a thing recursively, the same output as seen in btrfs sub list -a can be produced.

class btrfs.ctree.FileExtentItem(header, data)

Bases: ItemData

Object representation of btrfs_file_extent_item.

For every piece of a file, the FileExtentItem points to the data extent where the actual data is stored. A FileExtentItem does not have to reference a complete extent. It can also use part of it.

• Tree: FS_TREE_OBJECTID (5) or any other subvolume tree.

• Key objectid: Inode number of the file.

• Key type: EXTENT_DATA_KEY (108)

• Key offset: Logical offset in the file where the referenced data appears.

Variables:

• objectid (int) – The inode number of the file. (taken from the objectid field of the item key).

• logical_offset (int) – Logical offset in the file where the referenced data appears. (taken from the offset field of the item key).

• generation (int) – Generation of the filesystem when this file extent was created.

• ram_bytes (int) – Upper limit on the memory needed in bytes to store the extent after decompression.

• compression (int) – Compression type, see below.

The compression field can have one of the following values (available as attribute of this module):

• COMPRESS_NONE

• COMPRESS_ZLIB

• COMPRESS_LZO

• COMPRESS_ZSTD

Variables:

• encryption (int) – Not used Encryption type, always 0.

• other_encoding (int) – Not used

• type (int) – Type of extent, see below.

The extent type can be one of the following:

• FILE_EXTENT_INLINE: This is an inline extent. The data is stored inside

  the metadata leaf, right after the type field.

• FILE_EXTENT_REG: This is a regular extent.

• FILE_EXTENT_PREALLOC: Preallocated extent (for which no actual data is

  written yet).

If the extent type is FILE_EXTENT_INLINE, the following fields are not available:

Variables:

• disk_bytenr (int) – Virtual address of the data extent we reference a range from.

• disk_num_bytes (int) – Size of the data extent we reference a range from.

• offset (int) – The offset inside the data extent where the data we need starts.

• num_bytes (int) – The amount of bytes to be used from that offset onwards.

This means that (disk_bytenr EXTENT_ITEM disk_num_bytes) is the tree key of the extent item in the extent tree. Also, remember that these numbers will always be multiples of disk block sizes, because that’s how it gets cowed. We don’t just use 1 or 2 bytes from another extent.

class btrfs.ctree.FreeSpaceInfo(header, data)

Bases: ItemData

Object representation of struct btrfs_free_space_info.

The free space tree contains a free space info item for every block group.

• Tree: FREE_SPACE_TREE_OBJECTID (10).

• Key objectid: Virtual address.

• Key type: FREE_SPACE_INFO_KEY (198)

• Key offset: Block Group length.

Variables:

• vaddr (int) – Virtual address. (taken from the objectid field of the item key)

• length (int) – Block Group length. (taken from the offset field of the item key)

• extent_count (int) – Amount of free space extents in the Block Group.

• flags (int) – A flag indicating if the free space for this Block Group is stored as bitmap. The flag is FREE_SPACE_USING_BITMAPS, available as attribute of this module.

class btrfs.ctree.FreeSpaceExtent(header, data)

Bases: ItemData

Object representation for free space extent information.

• Tree: FREE_SPACE_TREE_OBJECTID (10).

• Key objectid: Virtual address of the start of the free space.

• Key type: FREE_SPACE_EXTENT_KEY (199)

• Key offset: Length of the free space.

Note that this metadata object type does not have actual item data. All needed information is encoded in the item key.

Variables:

• vaddr (int) – Virtual address of the start of the free space. (taken from the objectid field of the item key)

• length (int) – Length of the free space. (taken from the offset field of the item key)

class btrfs.ctree.FreeSpaceBitmap(header, data)

Bases: ItemData

Object representation for free space bitmap information.

• Tree: FREE_SPACE_TREE_OBJECTID (10).

• Key objectid: Virtual address of the start of the free space bitmap.

• Key type: FREE_SPACE_BITMAP_KEY (200)

• Key offset: Length of the covered virtual address space.

Variables:

• vaddr (int) – Virtual address of the start of the free space bitmap. (taken from the objectid field of the item key)

• length (int) – Length of the covered virtual address space. (taken from the offset field of the item key)

• bitmap (bytes) – The free space bitmap.

unpack(sectorsize)

Unpack the free space bitmap.

Parameters:

sectorsize (int) – sectorsize property of the filesystem.

Returns:

A generator of btrfs.free_space_tree.FreeSpaceExtent tuples.

Return type:

Iterator[btrfs.free_space_tree.FreeSpaceExtent]

class btrfs.ctree.OrphanItem(header, _)

Bases: ItemData

Object representation for orphan item information from the root tree.

• Tree: ROOT_TREE_OBJECTID (1).

• Key objectid: ORPHAN_OBJECTID (-5).

• Key type: ORPHAN_ITEM_KEY (48).

• Key offset: objectid of the item in the root tree that is orphaned and

  needs to be cleaned up.

Variables:

objectid (int) – objectid of the item in the root tree that is orphaned and needs to be cleaned up. (taken from the offset field of the item key)

class btrfs.ctree.NotImplementedItem(header, data)

Bases: ItemData

Placeholder object for metadata item types that have not been implemented yet.

class btrfs.ctree.UnknownItem(header, data)

Bases: ItemData

Placeholder object for metadata item types that are unrecognized.

btrfs.ctree.classify(header, data)

Convenience function to automatically convert an item header and data into one of the object types in this module.

Parameters:

• header (btrfs.ioctl.SearchHeader) – Search header.

• data (bytes) – Item data.

Returns:

Object representing the metadata item.

Return type:

Subclass of ItemData

Example:

>>> with btrfs.FileSystem('/') as fs:
...     chunk_tree_objects = btrfs.ioctl.search_v2(fs.fd, 3)
...     btrfs.utils.pretty_print(
...         (btrfs.ctree.classify(header, data)
...          for header, data in chunk_tree_objects)
...     )

The search function returns a generator, which we name chunk_tree_objects. The pretty printer can handle any iterable, so the above fragment will, in a ‘streaming’ way, dump the chunk tree on the screen.

btrfs.free_space_tree module

btrfs.fs_usage module

btrfs.ioctl module

btrfs.utils module

btrfs.volumes module

Module contents