• Stars
    star
    334
  • Rank 126,264 (Top 3 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created over 2 years ago
  • Updated 4 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

pytest plugin for easy integration of memray memory profiler

logo

pytest-memray

PyPI PyPI - Implementation PyPI - Python Version PyPI - Downloads PyPI - License Tests Code Style

pytest-memray is a pytest plugin for easy integration of memray.

Installation

pytest-memray requires Python 3.8 or higher and can be easily installed using most common Python packaging tools. We recommend installing the latest stable release from PyPI with pip:

pip install pytest-memray

Documentation

You can find the latest documentation available here.

Quick introduction

To use the plugin in a pytest run, simply add --memray to the command line invocation:

pytest --memray tests

After the test suite runs you'll see a memory report printed:

=================================== test session starts ====================================
platform linux -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0
cachedir: /v/.pytest_cache
rootdir: /w
plugins: memray-1.1.0
collected 2 items

demo/test_ok.py .M                                                                   [100%]

========================================= FAILURES =========================================
____________________________________ test_memory_exceed ____________________________________
Test was limited to 100.0KiB but allocated 117.2KiB
------------------------------------ memray-max-memory -------------------------------------
Test is using 117.2KiB out of limit of 100.0KiB
List of allocations:
	- <listcomp>:/w/demo/test_ok.py:17 -> 117.2KiB

====================================== MEMRAY REPORT =======================================
Allocations results for demo/test_ok.py::test_memory_exceed

	 πŸ“¦ Total memory allocated: 117.2KiB
	 πŸ“ Total allocations: 30
	 πŸ“Š Histogram of allocation sizes: |β–ˆ|
	 πŸ₯‡ Biggest allocating functions:
		- <listcomp>:/w/demo/test_ok.py:17 -> 117.2KiB


Allocations results for demo/test_ok.py::test_track

	 πŸ“¦ Total memory allocated: 54.9KiB
	 πŸ“ Total allocations: 71
	 πŸ“Š Histogram of allocation sizes: |β–ˆ   β–…    |
	 πŸ₯‡ Biggest allocating functions:
		- test_track:/w/demo/test_ok.py:12 -> 39.1KiB
		- _compile_bytecode:<frozen importlib._bootstrap_external>:672 -> 7.2KiB
		- _call_with_frames_removed:<frozen importlib._bootstrap>:241 -> 4.7KiB
		- _call_with_frames_removed:<frozen importlib._bootstrap>:241 -> 1.8KiB
		- _is_marked_for_rewrite:/v/lib/python3.10/site-packages/_pytest/assertion/rewrite.py:240 -> 1.1KiB


================================= short test summary info ==================================
MEMORY PROBLEMS demo/test_ok.py::test_memory_exceed
=============================== 1 failed, 1 passed in 0.01s ================================

Configuration - CLI flags

  • --memray - activate memray tracking
  • --most-allocations=MOST_ALLOCATIONS - show the N tests that allocate most memory (N=0 for all)
  • --hide-memray-summary - hide the memray summary at the end of the execution
  • --memray-bin-path - path where to write the memray binary dumps (by default a temporary folder)
  • --memray-bin-prefix - prefix to use for the binary dump (by default a random UUID4 hex)
  • --stacks=STACKS - Show the N stack entries when showing tracebacks of memory allocations
  • --native - Show native frames when showing tracebacks of memory allocations (will be slower)
  • --trace-python-allocators - Record allocations made by the Pymalloc allocator (will be slower)

Configuration - INI

  • memray(bool) - activate memray tracking
  • most-allocations(string) - show the N tests that allocate most memory (N=0 for all)
  • hide_memray_summary(bool) - hide the memray summary at the end of the execution
  • stacks(int) - Show the N stack entries when showing tracebacks of memory allocations
  • native(bool)- Show native frames when showing tracebacks of memory allocations (will be slower)
  • trace_python_allocators - Record allocations made by the Pymalloc allocator (will be slower)

License

pytest-memray is Apache-2.0 licensed, as found in the LICENSE file.

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behavior which you have experienced in the project, please contact us at [email protected].

Security Policy

If you believe you have identified a security vulnerability in this project, please send email to the project team at [email protected], detailing the suspected issue and any methods you've found to reproduce it.

Please do NOT open an issue in the GitHub repository, as we'd prefer to keep vulnerability reports private until we've had an opportunity to review and address them.

Contributing

We welcome your contributions to help us improve and extend this project!

Below you will find some basic steps required to be able to contribute to the project. If you have any questions about this process or any other aspect of contributing to a Bloomberg open source project, feel free to email [email protected], and we'll get your questions answered as quickly as we can.

Contribution Licensing

Since this project is distributed under the terms of an open source license, contributions that you make are licensed under the same terms. In order for us to be able to accept your contributions, we will need explicit confirmation from you that you are able and willing to provide them under these terms, and the mechanism we use to do this is called a Developer's Certificate of Origin (DCO). This is very similar to the process used by the Linux(R) kernel, Samba, and many other major open source projects.

To participate under these terms, all that you must do is include a line like the following as the last line of the commit message for each commit in your contribution:

Signed-Off-By: Random J. Developer <[email protected]>

The simplest way to accomplish this is to add -s or --signoff to your git commit command.

You must use your real name (sorry, no pseudonyms, and no anonymous contributions).

Steps

  • Create an Issue, selecting 'Feature Request', and explain the proposed change.
  • Follow the guidelines in the issue template presented to you.
  • Submit the Issue.
  • Submit a Pull Request and link it to the Issue by including "#" in the Pull Request summary.

Development

The project requires a Linux OS to work. To set up a DEV environment use tox (or directly the make targets). You can use Docker to run the test suite on non Linux as in (you can parametrize tox by passing additional arguments at the end):

docker-compose run --rm test tox

More Repositories

1

memray

Memray is a memory profiler for Python
Python
13,044
star
2

blazingmq

A modern high-performance open source message queuing system
C++
2,549
star
3

goldpinger

Debugging tool for Kubernetes which tests and displays connectivity between nodes in the cluster.
JavaScript
2,489
star
4

bde

Basic Development Environment - a set of foundational C++ libraries used at Bloomberg.
C++
1,542
star
5

comdb2

Bloomberg's distributed RDBMS
C
1,340
star
6

pystack

πŸ” 🐍 Like pstack but for Python!
Python
991
star
7

xcdiff

A tool which helps you diff xcodeproj files.
Swift
916
star
8

quantum

Powerful multi-threaded coroutine dispatcher and parallel execution engine
C++
573
star
9

ipydatagrid

Fast Datagrid widget for the Jupyter Notebook and JupyterLab
TypeScript
510
star
10

ts-blank-space

A small, fast, pure JavaScript type-stripper that uses the official TypeScript parser.
TypeScript
484
star
11

foml

Foundations of Machine Learning
Handlebars
334
star
12

python-github-webhook

A framework for writing webhooks for GitHub, in Python.
Python
276
star
13

chromium.bb

Chromium source code and modifications
267
star
14

koan

A word2vec negative sampling implementation with correct CBOW update.
C++
260
star
15

blpapi-node

Bloomberg Open API module for node.js
C++
243
star
16

chef-bcpc

Bloomberg Clustered Private Cloud distribution
Python
228
star
17

phabricator-tools

Phabricator Tools
Python
221
star
18

scatteract

Project which implements extraction of data from scatter plots
Jupyter Notebook
209
star
19

stricli

Build complex CLIs with type safety and no dependencies
TypeScript
180
star
20

pasta-sourcemaps

Pretty (and) Accurate Stack Trace Analysis is an extension to the JavaScript source map format that allows for accurate function name decoding.
TypeScript
166
star
21

record-tuple-polyfill

A polyfill for the ECMAScript Record and Tuple proposal.
JavaScript
162
star
22

collectdwin

CollectdWin - a system statistics collection daemon for Windows, inspired by 'collectd'
C#
123
star
23

clangmetatool

A framework for reusing code in Clang tools
C++
119
star
24

kubernetes-cluster-cookbook

Ruby
100
star
25

quant-research

A collection of projects published by Bloomberg's Quantitative Finance Research team.
Jupyter Notebook
100
star
26

blpapi-http

HTTP wrapper for Bloomberg Open API
TypeScript
83
star
27

dataless-model-merging

Code release for Dataless Knowledge Fusion by Merging Weights of Language Models (https://openreview.net/forum?id=FCnohuR6AnM)
Python
79
star
28

amqpprox

An AMQP 0.9.1 proxy server, designed for use in front of an AMQP 0.9.1 compliant message queue broker such as RabbitMQ.
C++
74
star
29

ntf-core

Sockets, timers, resolvers, events, reactors, proactors, and thread pools for asynchronous network programming
C++
71
star
30

spire-tpm-plugin

Provides agent and server plugins for SPIRE to allow TPM 2-based node attestation.
Go
70
star
31

bde-tools

Tools for developing and building libraries modeled on BDE
Perl
67
star
32

repofactor

Tools for refactoring history of git repositories
Perl
63
star
33

chef-bach

Chef recipes for Bloomberg's deployment of Hadoop and related components
Ruby
61
star
34

rmqcpp

A batteries included C++ RabbitMQ Client Library/API.
C++
59
star
35

minilmv2.bb

Our open source implementation of MiniLMv2 (https://aclanthology.org/2021.findings-acl.188)
Python
59
star
36

wsk

A straightforward and maintainable build system from the Bloomberg Graphics team.
JavaScript
57
star
37

git-adventure-game

An adventure game to help people learn Git
Shell
54
star
38

attrs-strict

Provides runtime validation of attributes specified in Python 'attr'-based data classes.
Python
52
star
39

cnn-rnf

Convolutional Neural Networks with Recurrent Neural Filters
Python
51
star
40

corokafka

C++ Kafka coroutine library using Quantum dispatcher and wrapping CppKafka
C++
50
star
41

selekt

A Kotlin and familiar Android SQLite database library that uses encryption.
Kotlin
46
star
42

ppx_string_interpolation

PPX rewriter that enables string interpolation in OCaml
OCaml
45
star
43

bde_verify

Tool used to format, improve and verify code to BDE guidelines
C++
42
star
44

vault-auth-spire

vault-auth-spire is an authentication plugin for Hashicorp Vault which allows logging into Vault using a Spire provided SVID.
Go
41
star
45

spark-flow

Library for organizing batch processing pipelines in Apache Spark
Scala
41
star
46

startup-python-bootcamp

35
star
47

p1160

P1160 Add Test Polymorphic Memory Resource To Standard Library
C++
35
star
48

bbit-learning-labs

Learning labs curated by BBIT
JavaScript
34
star
49

chef-umami

A tool to automatically generate test code for Chef cookbooks and policies.
Ruby
33
star
50

pycsvw

A tool to read CSV files with CSVW metadata and transform them into other formats.
Python
32
star
51

bde-allocator-benchmarks

A set of benchmarking tools used to quantify the performance of BDE-style polymorphic allocators.
C++
31
star
52

blpapi-hs

Haskell interface to BLPAPI
Haskell
30
star
53

rwl-bench

A set of benchmark tools for reader/writer locks.
C++
28
star
54

entsum

Open Source / ENTSUM: A Data Set for Entity-Centric Extractive Summarization
Jupyter Notebook
28
star
55

consul-cluster-cookbook

Wrapper cookbook which installs and configures a Consul cluster.
Ruby
26
star
56

python-comdb2

Python API to Bloomberg's comdb2 database.
Python
26
star
57

kbir_keybart

Experimental code used in pre-training the KBIR and KeyBART models
Python
26
star
58

blazingmq-sdk-java

Java SDK for BlazingMQ, a modern high-performance open source message queuing system.
Java
26
star
59

presto-accumulo

Presto Accumulo Integration
Java
25
star
60

sgtb

Structured Gradient Tree Boosting
Python
25
star
61

blazingmq-sdk-python

Python SDK for BlazingMQ, a modern high-performance open source message queuing system.
Python
24
star
62

docket

Tool to make running test suites easier, using docker-compose.
Go
22
star
63

jupyterhub-kdcauthenticator

A Kerberos authenticator module for the JupyterHub platform
Python
22
star
64

tzcron

A parser of cron-style scheduling expressions.
Python
20
star
65

constant.js

Immutable/Constant Objects for JavaScript
JavaScript
20
star
66

redis-cookbook

A set of Chef recipes for installing and configuring Redis.
HTML
19
star
67

go-testgroup

Helps you organize tests in Go programs into groups.
Go
19
star
68

MixCE-acl2023

Implementation of MixCE method described in ACL 2023 paper by Zhang et al.
Python
19
star
69

userchroot

A tool to allow controlled access to 'chroot' functionality by users without root permissions
C
19
star
70

sable

Database migration tool for Marten.
C#
19
star
71

nginx-cookbook

A set of Chef recipes for installing and configuring Nginx.
Ruby
17
star
72

zookeeper-cookbook

A set of Chef recipes for installing and configuring Apache Zookeeper.
Ruby
17
star
73

mynexttalk

16
star
74

chef-bcs

Bloomberg Cloud Storage Chef application
Ruby
16
star
75

git-adventure-game-builder

A set of tools for building a Git adventure game, to help people learn Git
Shell
15
star
76

vault-cluster-cookbook

Application cookbook which installs and configures Vault with Consul as a backend.
Ruby
15
star
77

emnlp20_depsrl

Research code and scripts used in the paper Semantic Role Labeling as Syntactic Dependency Parsing.
Python
14
star
78

coffeechat

A simple web application for arranging 'chats over coffee'.
TypeScript
12
star
79

k8eraid

A relatively simple, unified method for reporting on Kubernetes resource issues.
Go
12
star
80

pytest-pystack

Pytest plugin that runs PyStack on slow or hanging tests.
Python
12
star
81

hackathon-aws-cluster

HTML
11
star
82

fast-noise-aware-topic-clustering

Research code and scripts used in the Silburt et al. (2021) EMNLP 2021 paper 'FANATIC: FAst Noise-Aware TopIc Clustering'
Python
10
star
83

emnlp21_fewrel

Code to reproduce the results of the paper 'Towards Realistic Few-Shot Relation Extraction' (EMNLP 2021)
Python
10
star
84

mastering-difficult-conversations

Plan It, Say It, Nail It: Mastering Difficult Conversations
10
star
85

wsk-notify

Simple, customizable console notifications.
JavaScript
10
star
86

jenkins-cluster-cookbook

Ruby
9
star
87

decorator-taxonomy

A taxonomy of Python decorator types.
HTML
9
star
88

tdd-labs

Problems and Solutions for Test-Driven-Development training
JavaScript
9
star
89

argument-relation-transformer-acl2022

This repository contains code for our ACL 2022 Findings paper `Efficient Argument Structure Extraction with Transfer Learning and Active Learning`. We implement an argument structure extraction method based on a pre-trained Transformer model.`
Python
9
star
90

sigir2018-kg-contextualization

8
star
91

bloomberg.github.io

Source code for the https://bloomberg.github.io site
HTML
8
star
92

locking_resource-cookbook

Chef cookbook for serializing access to resources
Ruby
7
star
93

datalake-query-ingester

Python
7
star
94

cobbler-cookbook

A Chef cookbook for installing and maintaining Cobbler
Ruby
7
star
95

p2473

Example code for WG21 paper P2473
Perl
6
star
96

collectd-cookbook

Ruby
6
star
97

Catalyst-Authentication-Credential-GSSAPI

A module that provides integration of the Catalyst web application framework with GSSAPI/SPNEGO HTTP authentication.
Perl
6
star
98

bob-bot

Java
5
star
99

.github

Organization-wide community files
5
star
100

jenkins-procguard

Perl
5
star