• Stars
    star
    277
  • Rank 148,875 (Top 3 %)
  • Language Verilog
  • License
    Other
  • Created over 5 years ago
  • Updated over 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

Defines a lossless compressed data format that is independent of CPU type, operating system, file system, and character set, and is suitable for compression using the XP10 algorithm.

Project Zipline

Introduction

Project Zipline is a program to accelerate innovation in lossless compression. With this release Microsoft is making available:

  • A new compression format called XP10 which is tailored for modern cloud datasets.
  • All specifications for the entire pipeline.
  • All RTL to support implementation for the entire pipeline.
  • A testbench environment specifically developed for the VCS simulator.

Project Zipline is open-sourced under the MIT License, see the LICENSE file.

Specifications

The following Project Zipline specifications have been included under the β€œspecs” directory in this repository:

  • Project_Zipline_Compression_Specification
  • Project_Zipline_Huffman_Encoder_Micro_Architecture_Specification
  • Project_Zipline_Comp_lz77_Micro_Architecture_Specification
  • Project_Zipline_Crypto_Engine_Micro_Architecture_Specification
  • Project_Zipline_Decompression_Top_Micro_Architecture_Specification
  • Project_Zipline_Keyblob_Micro_Architecture_Specification
  • Project_Zipline_KME_Micro_Architecture_Specification
  • Project_Zipline_Prefix_Attach_Micro_Architecture_Specification
  • Project_Zipline_Prefix_Micro_Architecture_Specification
  • Project_Zipline_SSB_Micro_Architecture_Specification
  • Project_Zipline_Top_Micro_Architecture_Specification

Simulation Notes

Edit the file ./zipline.setup for VCS, SYNTH, and VERDI environment settings.

Then source the file as follows:

source zipline.setup

Go to either the CCE_64, CDD_64, or KME run directory:

cd dv/CCE_64/run

cd dv/CDD_64/run

cd dv/KME/run

To build the simv executable:

make build_simv

Example for simulating a test:

make run_simv TESTNAME=xp10

Example for simulating a test with waves (.vpd):

make run_simv TESTNAME=xp10 WAVES=1

Example for simulating a test with Verdi waves (.fsdb):

make run_simv TESTNAME=xp10 VERDI_WAVES=1

All of the tests for the CCE_64, CDD_64, and KME can be run via a regress script in the associated "run" directory:

dv/CCE_64/run/regress

dv/CDD_64/run/regress

dv/KME/run/regress

All test files are located in the associated engine directory "tests". Each directory also contains a README file with a description of each test.

dv/CCE_64/tests

dv/CDD_64/tests

dv/KME/tests

A description of the programmable registers in the CCE/CDD/KME can be found in the register_doc directory. Please read register_doc/README for more details.

KME Notes

The KME RTL (rtl/cr_kme) has been modified to remove the following modules:

  • AES engine in the random GUID generator
  • SHA engines within the KDF function
  • AES engine within the Key Decryption logic

With these reductions, the RTL only supports Key Types 1-6 without KDF and Key Type 0 without encryption/authentication

Additionally, the KME output has been reduced to support a single engine.

All modifications are identified with "KME_MODIFICATION_NOTE" comments in the RTL code.

Synthesis Notes

Note: For synthesis, please edit syn/Makefile to choose an LSF or dedicated machine with at least 16G of memory. Edit the contents of this line, before "dc_shell":

bsub -Is -q irv-cpx-M16 -R "rusage [mem=16000]" dc_shell -f syn_example.tcl | tee ./LOGS/syn_eample.tcl.date '+%m%d.%H:%M'.log

To run synthesis for CDD

cd syn/cr_cddip

make syn

To run synthesis for CCE

cd syn/cr_cceip_64

make syn

To run synthesis for KME

cd syn/cr_kme

make syn

More Repositories

1

Time-Appliance-Project

Develop an end-to-end hypothetical reference model, network architectures, precision time tools, performance objectives and the methods to distribute, operate, monitor time synchronization within data center and much more...
C
1,340
star
2

onie

Open Network Install Environment
Shell
601
star
3

OpenNetworkLinux

Open Network Linux - An Operating System for Bare Metal Switches
C
570
star
4

SAI

Switch Abstraction Interface
Python
475
star
5

Project_Olympus

313
star
6

ocp-diag-pcicrawler

pcicrawler is a Python based command line interface tool which can be used to display, filter and export information about PCI (Peripheral Component Interconnect) or PCIe buses and devices, as well as PCI topology.
Python
193
star
7

OCP-OSF-Aptio_Community_Edition

TianoCore/EDKII for UEFI implementation
C
117
star
8

OCP-HM-MegaRAC_Community_Edition

AMIs Community Edition for OpenBMC
Python
98
star
9

CADCloud

CADCloud is a cloud based sharing infrastructure dedicates to CAD engineers. It allows users to share design files, browse them interactively through a simple web interface, and support native file format from popular open source tools.
JavaScript
88
star
10

Security

Security Project
86
star
11

OpenSystemFirmware

Open Systems Firmware (OSF)
Shell
68
star
12

oom

Python
66
star
13

OCP-NET-Falcon

35
star
14

ocp-diag-core

The main Test & Validation specification and tools
Go
30
star
15

zaius-barreleye-g2

OpenPOWER / Open Compute Server, based upon POWER9
28
star
16

ODSA-BoW

Repo for all activity related to the ODSA Bunch of Wires Specification
23
star
17

HWMgmt-DeviceMgr-DeviceManager

Device Manager collects device data and notifications from each device, and make the data available on a predetermined output bus for consumers.
Go
23
star
18

OSF-OSFCI

Open Source Firmware Continuous Integration source code
Go
22
star
19

OCP-OSF-Tektagon_Community_Edition

Hardware Root of Trust
C
20
star
20

Rack-Manager

The base repo for the OCP OpenRMC Initiative
C++
18
star
21

ocp-diag-pci_lmt

PCIe Lane Margining Tool (LMT)
Python
16
star
22

SAI-Challenger

Python
15
star
23

HWMgmt-OCP-Profiles

A place where all the OCP profiles are a placed. Issues can be submitted/resolved and modifications can be reviewed/merged
HTML
14
star
24

HWMgmt-DeviceMgr-PSME

This repository is a part of the "Device Manager" repository
C++
13
star
25

ODSA-CDXML

TBD
Python
11
star
26

RunBMC

RunBMC specification defines the interface between the Baseboard Management Controller(BMC) subsystem and OCP hardware platforms, such as network or compute motherboards.
11
star
27

mion

Base OS switch for networking
8
star
28

OCP-Security-SAFE

Python
8
star
29

ocp-diag-core-python

Pure python API for the OCP Test & Validation project
Python
8
star
30

ODSA-Link-Layer

New workstream within OCP's Open Domain Specific Architecture Project working on Link Layer
8
star
31

ocp-diag-autoval-ssd

Python
8
star
32

ocp-diag-autoval

Python
8
star
33

FP8

6
star
34

HWMgmt-Module-DCSCM-LTPI

DC-SCM LTPI Reference Implementation
SystemVerilog
6
star
35

ocp-diag-ctam

Python
5
star
36

Discovery

Work in progress for hardware rack discovery
5
star
37

PowerAPI

PowerAPI Reference Implementation and Plugins for OCP Community
C++
4
star
38

Project_Olympus_rack_manager

Python
4
star
39

OCP-Accepted-Ampere-Computing

4
star
40

OCP-STO-HDD-NVMe_Telematics

Python
4
star
41

ocp-diag-sat

C++
3
star
42

ocp-diag-storage-staging

organize code before breaking up diags into their respective repositories
3
star
43

ocp-diag-memtester

Python scripts that turn memtester into an OCP-compliant diagnostic
Python
3
star
44

OAMTool

A tool for managing OA modules
3
star
45

OCP-Ready-Facility-Recognition-Program

3
star
46

OCP-Public

This is the public OCP repository
3
star
47

OCP-Software-GitHub-Process

Defining processes and guidelines
3
star
48

HWMgmt-OpenRMC-contrib-Inspur

Vue
3
star
49

OCP-SC-Steering-Committee

Repository for the IC's to manage the work items
2
star
50

xAI

Serving series of APIs and defines standard API for platform abstraction interface (PAI), and Chassis abstraction interface (CHAI)
2
star
51

OCP-HM-SMC-Satellite_Management_Controller

2
star
52

ocp-diag-core-cpp

C++
2
star
53

OCP-HM-RAS_API

2
star
54

OCP-Regional-Europe

1
star
55

OCP-CE-IC-Immersion_Cooling

1
star
56

OCP-Regional-Japan

1
star
57

OCP-SVR-NIC-Network_Interface_Card

1
star
58

OCP-Regional-Korea

1
star
59

OCP-Regional-Taiwan

1
star
60

OCP-CE-HR-Heat_Reuse

1
star
61

OCP-NET-Networking

1
star
62

OCP-RP-Rack-and-Power

1
star
63

OCP-SI-Strategic_Initiatives

1
star
64

Project_Olympus_meta-rackmanager

BitBake
1
star
65

OCP-SUS-Sustainability

1
star
66

OCP-Telco

1
star
67

OCP-Archive

1
star
68

OCP-DCF-MDC-Modular_Data_Center

1
star
69

OCP-SUS-PT-Power_Telemetry

1
star
70

OCP-FTI-SDM-Software_Defined_Memory

1
star
71

OCP-SVR-OAI-Open_Accelerator_Infrastructure

1
star
72

ODSA-PoC

Repo for all activity related to the ODSA Proof-of-Concept
1
star
73

OCP-STO-Storage

1
star
74

OCP-FTI-AI_HW_SW_Design_Collaboration

1
star
75

OCP-HM-FMFM-Fleetscale_Memory_Fault_Management

1
star
76

OCP-NET-NIC3-TTF-MCU

OCP NIC 3.0 Thermal test fixture MCU
C++
1
star
77

OCP-Regional-PRC-Peoples_Republic_of_China

1
star
78

OCP-NET-ECS-Enterprise_Connectivity_Solutions

1
star
79

OCP-SVR-MHS

1
star
80

OCP-SVR-ODSA-3DK-MDK

1
star
81

OCP-SVR-ODSA-3DK-PADK

1
star
82

OCP-HM-CSM-Cloud_Service_Model

1
star
83

OCP-DCF-Data_Center_Facility

1
star
84

Project-Olympus-rackmanager-build

Shell
1
star
85

OCP-SVR-Server

1
star
86

OCP-FTI-Future_Technologies_Initiative

1
star
87

OCP-CE-DHE-Door_Heat_Exchange

1
star
88

OCP-DCF-OTS-Operation_Technology_Security

1
star
89

OCP-NET-NS-NIC_Software

1
star
90

Project-Olympus-Rackmanager-bsp

1
star
91

OCP-SVR-HPC-High_Performance_Computing

1
star
92

OCP-SVR-PCIe_Extended_Connectivity_Requirements

1
star
93

ODSA

1
star
94

OCP-SVR-CMS-Composable_Memory_System

1
star
95

OCP-CE-CP-Cold_Plate

1
star
96

ocp-diag-ssd-qual

Python
1
star