• Stars
    star
    318
  • Rank 128,970 (Top 3 %)
  • Language
    Python
  • License
    Apache License 2.0
  • Created about 2 years ago
  • Updated 2 months ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

pytest plugin for easy integration of memray memory profiler

logo

pytest-memray

PyPI PyPI - Implementation PyPI - Python Version PyPI - Downloads PyPI - License Tests Code Style

pytest-memray is a pytest plugin for easy integration of memray.

Installation

pytest-memray requires Python 3.8 or higher and can be easily installed using most common Python packaging tools. We recommend installing the latest stable release from PyPI with pip:

pip install pytest-memray

Documentation

You can find the latest documentation available here.

Quick introduction

To use the plugin in a pytest run, simply add --memray to the command line invocation:

pytest --memray tests

After the test suite runs you'll see a memory report printed:

=================================== test session starts ====================================
platform linux -- Python 3.10.4, pytest-7.1.2, pluggy-1.0.0
cachedir: /v/.pytest_cache
rootdir: /w
plugins: memray-1.1.0
collected 2 items

demo/test_ok.py .M                                                                   [100%]

========================================= FAILURES =========================================
____________________________________ test_memory_exceed ____________________________________
Test was limited to 100.0KiB but allocated 117.2KiB
------------------------------------ memray-max-memory -------------------------------------
Test is using 117.2KiB out of limit of 100.0KiB
List of allocations:
	- <listcomp>:/w/demo/test_ok.py:17 -> 117.2KiB

====================================== MEMRAY REPORT =======================================
Allocations results for demo/test_ok.py::test_memory_exceed

	 πŸ“¦ Total memory allocated: 117.2KiB
	 πŸ“ Total allocations: 30
	 πŸ“Š Histogram of allocation sizes: |β–ˆ|
	 πŸ₯‡ Biggest allocating functions:
		- <listcomp>:/w/demo/test_ok.py:17 -> 117.2KiB


Allocations results for demo/test_ok.py::test_track

	 πŸ“¦ Total memory allocated: 54.9KiB
	 πŸ“ Total allocations: 71
	 πŸ“Š Histogram of allocation sizes: |β–ˆ   β–…    |
	 πŸ₯‡ Biggest allocating functions:
		- test_track:/w/demo/test_ok.py:12 -> 39.1KiB
		- _compile_bytecode:<frozen importlib._bootstrap_external>:672 -> 7.2KiB
		- _call_with_frames_removed:<frozen importlib._bootstrap>:241 -> 4.7KiB
		- _call_with_frames_removed:<frozen importlib._bootstrap>:241 -> 1.8KiB
		- _is_marked_for_rewrite:/v/lib/python3.10/site-packages/_pytest/assertion/rewrite.py:240 -> 1.1KiB


================================= short test summary info ==================================
MEMORY PROBLEMS demo/test_ok.py::test_memory_exceed
=============================== 1 failed, 1 passed in 0.01s ================================

Configuration - CLI flags

  • --memray - activate memray tracking
  • --most-allocations=MOST_ALLOCATIONS - show the N tests that allocate most memory (N=0 for all)
  • --hide-memray-summary - hide the memray summary at the end of the execution
  • --memray-bin-path - path where to write the memray binary dumps (by default a temporary folder)
  • --memray-bin-prefix - prefix to use for the binary dump (by default a random UUID4 hex)
  • --stacks=STACKS - Show the N stack entries when showing tracebacks of memory allocations
  • --native - Show native frames when showing tracebacks of memory allocations (will be slower)
  • --trace-python-allocators - Record allocations made by the Pymalloc allocator (will be slower)

Configuration - INI

  • memray(bool) - activate memray tracking
  • most-allocations(string) - show the N tests that allocate most memory (N=0 for all)
  • hide_memray_summary(bool) - hide the memray summary at the end of the execution
  • stacks(int) - Show the N stack entries when showing tracebacks of memory allocations
  • native(bool)- Show native frames when showing tracebacks of memory allocations (will be slower)
  • trace_python_allocators - Record allocations made by the Pymalloc allocator (will be slower)

License

pytest-memray is Apache-2.0 licensed, as found in the LICENSE file.

Code of Conduct

This project has adopted a Code of Conduct. If you have any concerns about the Code, or behavior which you have experienced in the project, please contact us at [email protected].

Security Policy

If you believe you have identified a security vulnerability in this project, please send email to the project team at [email protected], detailing the suspected issue and any methods you've found to reproduce it.

Please do NOT open an issue in the GitHub repository, as we'd prefer to keep vulnerability reports private until we've had an opportunity to review and address them.

Contributing

We welcome your contributions to help us improve and extend this project!

Below you will find some basic steps required to be able to contribute to the project. If you have any questions about this process or any other aspect of contributing to a Bloomberg open source project, feel free to email [email protected], and we'll get your questions answered as quickly as we can.

Contribution Licensing

Since this project is distributed under the terms of an open source license, contributions that you make are licensed under the same terms. In order for us to be able to accept your contributions, we will need explicit confirmation from you that you are able and willing to provide them under these terms, and the mechanism we use to do this is called a Developer's Certificate of Origin (DCO). This is very similar to the process used by the Linux(R) kernel, Samba, and many other major open source projects.

To participate under these terms, all that you must do is include a line like the following as the last line of the commit message for each commit in your contribution:

Signed-Off-By: Random J. Developer <[email protected]>

The simplest way to accomplish this is to add -s or --signoff to your git commit command.

You must use your real name (sorry, no pseudonyms, and no anonymous contributions).

Steps

  • Create an Issue, selecting 'Feature Request', and explain the proposed change.
  • Follow the guidelines in the issue template presented to you.
  • Submit the Issue.
  • Submit a Pull Request and link it to the Issue by including "#" in the Pull Request summary.

Development

The project requires a Linux OS to work. To set up a DEV environment use tox (or directly the make targets). You can use Docker to run the test suite on non Linux as in (you can parametrize tox by passing additional arguments at the end):

docker-compose run --rm test tox

More Repositories

1

memray

Memray is a memory profiler for Python
Python
12,679
star
2

blazingmq

A modern high-performance open source message queuing system
C++
2,490
star
3

goldpinger

Debugging tool for Kubernetes which tests and displays connectivity between nodes in the cluster.
JavaScript
2,457
star
4

bde

Basic Development Environment - a set of foundational C++ libraries used at Bloomberg.
C++
1,542
star
5

comdb2

Bloomberg's distributed RDBMS
C
1,311
star
6

pystack

πŸ” 🐍 Like pstack but for Python!
Python
962
star
7

xcdiff

A tool which helps you diff xcodeproj files.
Swift
909
star
8

quantum

Powerful multi-threaded coroutine dispatcher and parallel execution engine
C++
567
star
9

ipydatagrid

Fast Datagrid widget for the Jupyter Notebook and JupyterLab
TypeScript
510
star
10

foml

Foundations of Machine Learning
Handlebars
330
star
11

python-github-webhook

A framework for writing webhooks for GitHub, in Python.
Python
276
star
12

chromium.bb

Chromium source code and modifications
267
star
13

koan

A word2vec negative sampling implementation with correct CBOW update.
C++
261
star
14

blpapi-node

Bloomberg Open API module for node.js
C++
243
star
15

chef-bcpc

Bloomberg Clustered Private Cloud distribution
Python
228
star
16

phabricator-tools

Phabricator Tools
Python
221
star
17

scatteract

Project which implements extraction of data from scatter plots
Jupyter Notebook
208
star
18

record-tuple-polyfill

A polyfill for the ECMAScript Record and Tuple proposal.
JavaScript
162
star
19

pasta-sourcemaps

Pretty (and) Accurate Stack Trace Analysis is an extension to the JavaScript source map format that allows for accurate function name decoding.
TypeScript
160
star
20

collectdwin

CollectdWin - a system statistics collection daemon for Windows, inspired by 'collectd'
C#
123
star
21

clangmetatool

A framework for reusing code in Clang tools
C++
119
star
22

kubernetes-cluster-cookbook

Ruby
100
star
23

quant-research

A collection of projects published by Bloomberg's Quantitative Finance Research team.
Jupyter Notebook
98
star
24

blpapi-http

HTTP wrapper for Bloomberg Open API
TypeScript
83
star
25

dataless-model-merging

Code release for Dataless Knowledge Fusion by Merging Weights of Language Models (https://openreview.net/forum?id=FCnohuR6AnM)
Python
74
star
26

amqpprox

An AMQP 0.9.1 proxy server, designed for use in front of an AMQP 0.9.1 compliant message queue broker such as RabbitMQ.
C++
72
star
27

spire-tpm-plugin

Provides agent and server plugins for SPIRE to allow TPM 2-based node attestation.
Go
71
star
28

bde-tools

Tools for developing and building libraries modeled on BDE
Perl
67
star
29

ntf-core

Sockets, timers, resolvers, events, reactors, proactors, and thread pools for asynchronous network programming
C++
67
star
30

repofactor

Tools for refactoring history of git repositories
Perl
63
star
31

chef-bach

Chef recipes for Bloomberg's deployment of Hadoop and related components
Ruby
61
star
32

minilmv2.bb

Our open source implementation of MiniLMv2 (https://aclanthology.org/2021.findings-acl.188)
Python
59
star
33

wsk

A straightforward and maintainable build system from the Bloomberg Graphics team.
JavaScript
58
star
34

git-adventure-game

An adventure game to help people learn Git
Shell
57
star
35

corokafka

C++ Kafka coroutine library using Quantum dispatcher and wrapping CppKafka
C++
50
star
36

attrs-strict

Provides runtime validation of attributes specified in Python 'attr'-based data classes.
Python
50
star
37

cnn-rnf

Convolutional Neural Networks with Recurrent Neural Filters
Python
49
star
38

rmqcpp

A batteries included C++ RabbitMQ Client Library/API.
C++
46
star
39

selekt

A Kotlin and familiar Android SQLite database library that uses encryption.
Kotlin
45
star
40

ppx_string_interpolation

PPX rewriter that enables string interpolation in OCaml
OCaml
44
star
41

bde_verify

Tool used to format, improve and verify code to BDE guidelines
C++
42
star
42

vault-auth-spire

vault-auth-spire is an authentication plugin for Hashicorp Vault which allows logging into Vault using a Spire provided SVID.
Go
41
star
43

spark-flow

Library for organizing batch processing pipelines in Apache Spark
Scala
41
star
44

startup-python-bootcamp

35
star
45

chef-umami

A tool to automatically generate test code for Chef cookbooks and policies.
Ruby
34
star
46

p1160

P1160 Add Test Polymorphic Memory Resource To Standard Library
C++
34
star
47

pycsvw

A tool to read CSV files with CSVW metadata and transform them into other formats.
Python
32
star
48

bde-allocator-benchmarks

A set of benchmarking tools used to quantify the performance of BDE-style polymorphic allocators.
C++
31
star
49

blpapi-hs

Haskell interface to BLPAPI
Haskell
30
star
50

bbit-learning-labs

Learning labs curated by BBIT
Python
28
star
51

rwl-bench

A set of benchmark tools for reader/writer locks.
C++
28
star
52

entsum

Open Source / ENTSUM: A Data Set for Entity-Centric Extractive Summarization
Jupyter Notebook
28
star
53

consul-cluster-cookbook

Wrapper cookbook which installs and configures a Consul cluster.
Ruby
26
star
54

kbir_keybart

Experimental code used in pre-training the KBIR and KeyBART models
Python
26
star
55

presto-accumulo

Presto Accumulo Integration
Java
25
star
56

sgtb

Structured Gradient Tree Boosting
Python
25
star
57

blazingmq-sdk-java

Java SDK for BlazingMQ, a modern high-performance open source message queuing system.
Java
24
star
58

python-comdb2

Python API to Bloomberg's comdb2 database.
Python
23
star
59

jupyterhub-kdcauthenticator

A Kerberos authenticator module for the JupyterHub platform
Python
22
star
60

docket

Tool to make running test suites easier, using docker-compose.
Go
22
star
61

blazingmq-sdk-python

Python SDK for BlazingMQ, a modern high-performance open source message queuing system.
Python
21
star
62

tzcron

A parser of cron-style scheduling expressions.
Python
20
star
63

constant.js

Immutable/Constant Objects for JavaScript
JavaScript
20
star
64

go-testgroup

Helps you organize tests in Go programs into groups.
Go
19
star
65

redis-cookbook

A set of Chef recipes for installing and configuring Redis.
HTML
19
star
66

userchroot

A tool to allow controlled access to 'chroot' functionality by users without root permissions
C
19
star
67

nginx-cookbook

A set of Chef recipes for installing and configuring Nginx.
Ruby
17
star
68

MixCE-acl2023

Implementation of MixCE method described in ACL 2023 paper by Zhang et al.
Python
17
star
69

zookeeper-cookbook

A set of Chef recipes for installing and configuring Apache Zookeeper.
Ruby
17
star
70

mynexttalk

16
star
71

chef-bcs

Bloomberg Cloud Storage Chef application
Ruby
16
star
72

vault-cluster-cookbook

Application cookbook which installs and configures Vault with Consul as a backend.
Ruby
15
star
73

git-adventure-game-builder

A set of tools for building a Git adventure game, to help people learn Git
Shell
15
star
74

emnlp20_depsrl

Research code and scripts used in the paper Semantic Role Labeling as Syntactic Dependency Parsing.
Python
14
star
75

coffeechat

A simple web application for arranging 'chats over coffee'.
TypeScript
12
star
76

k8eraid

A relatively simple, unified method for reporting on Kubernetes resource issues.
Go
12
star
77

hackathon-aws-cluster

HTML
11
star
78

fast-noise-aware-topic-clustering

Research code and scripts used in the Silburt et al. (2021) EMNLP 2021 paper 'FANATIC: FAst Noise-Aware TopIc Clustering'
Python
10
star
79

emnlp21_fewrel

Code to reproduce the results of the paper 'Towards Realistic Few-Shot Relation Extraction' (EMNLP 2021)
Python
10
star
80

mastering-difficult-conversations

Plan It, Say It, Nail It: Mastering Difficult Conversations
10
star
81

wsk-notify

Simple, customizable console notifications.
JavaScript
10
star
82

jenkins-cluster-cookbook

Ruby
9
star
83

decorator-taxonomy

A taxonomy of Python decorator types.
HTML
9
star
84

pytest-pystack

Pytest plugin that runs PyStack on slow or hanging tests.
Python
9
star
85

tdd-labs

Problems and Solutions for Test-Driven-Development training
JavaScript
9
star
86

argument-relation-transformer-acl2022

This repository contains code for our ACL 2022 Findings paper `Efficient Argument Structure Extraction with Transfer Learning and Active Learning`. We implement an argument structure extraction method based on a pre-trained Transformer model.`
Python
9
star
87

sigir2018-kg-contextualization

8
star
88

bloomberg.github.io

Source code for the https://bloomberg.github.io site
HTML
8
star
89

locking_resource-cookbook

Chef cookbook for serializing access to resources
Ruby
7
star
90

datalake-query-ingester

Python
7
star
91

cobbler-cookbook

A Chef cookbook for installing and maintaining Cobbler
Ruby
7
star
92

p2473

Example code for WG21 paper P2473
Perl
6
star
93

collectd-cookbook

Ruby
6
star
94

Catalyst-Authentication-Credential-GSSAPI

A module that provides integration of the Catalyst web application framework with GSSAPI/SPNEGO HTTP authentication.
Perl
6
star
95

bob-bot

Java
5
star
96

.github

Organization-wide community files
5
star
97

jenkins-procguard

Perl
5
star
98

datalake-query-db-consumer

Python
4
star
99

datalake-metrics-db

Python
3
star
100

collectd_plugins-cookbook

Ruby
3
star