• Stars
    star
    176
  • Rank 216,987 (Top 5 %)
  • Language
  • License
    Apache License 2.0
  • Created almost 4 years ago
  • Updated about 1 month ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

alibabacloud-jindodata

AlibabaCloud JindoData

JindoData 介绍

JindoData 是阿里云开源大数据团队自研的数据湖存储加速套件,面向大数据和 AI 生态,为阿里云和业界主要数据湖存储系统提供全方位访问加速解决方案。JindoData 套件基于统一架构和内核实现,主要包括 JindoFS 存储系统(原 JindoFS Block 模式),JindoFSx 存储加速系统(原 JindoFS Cache 模式),JindoSDK 大数据万能 SDK 和全面兼容的生态工具(JindoFuse、JindoDistCp)、插件支持。

JindoData

主要组件

JindoFS 存储系统

基于阿里云 OSS 的云原生存储系统,二进制兼容 Apache HDFS,并且基本功能对齐,提供优化的 HDFS 使用和平迁体验。是原 JindoFS Block 模式的全新升级版本。 阿里云 OSS-HDFS 服务(JindoFS 服务) 是 JindoFS 存储系统在阿里云上的服务化部署形态,和阿里云 OSS 深度融合,开箱即用,无须在自建集群部署维护 JindoFS,免运维。OSS-HDFS 服务具体介绍请参考 OSS-HDFS服务概述

JindoFSx 存储加速系统

面向大数据和 AI 生态的云原生数据湖存储加速系统。为大数据和 AI 应用访问各种云存储提供访问加速,支持数据缓存,元数据缓存,P2P加速等功能。JindoFSx 支持管理多个后端存储系统,可以通过统一命名空间进行管理,也可以兼容各系统原生的访问协议,也支持为这些系统提供统一的权限管理。原生优化支持阿里云 OSS 和阿里云 OSS-HDFS 服务,同时也支持业界主要多云对象存储(AWS S3 等)和 Apache HDFS、NAS。是原 JindoFS Cache 模式的全新升级版本。

生态支持和工具

  • JindoSDK 支持。面向云时代的大数据 Hadoop SDK 和 HDFS 接口支持,内置优化访问阿里云 OSS,较 Hadoop 社区版本性能大幅提升;同时对接支持 JindoFS 存储系统包括服务、JindoFSx 存储加速系统;支持多云对象存储。
  • JindoShell CLI 支持。JindoData 除了对接支持 Hadoop/HDFS shell 命令,同时提供一套 JindoShell CLI 命令,从功能、性能上大幅扩展和优化一些数据访问操作。
  • JindoFuse POSIX 支持。JindoData 为阿里云 OSS、JindoFS 存储系统和服务、JindoFSx 存储加速系统提供的 POSIX 支持。
  • JindoDistCp 数据迁移支持。IDC 机房数据(HDFS)上云迁移和多云迁移利器,支持多种存储数据迁移到阿里云 OSS 和 JindoFS 服务,使用上类似Hadoop DistCp。
  • JindoTable 支持。结合计算引擎的使用推出的一套解决方案,支持 Spark、Hive、Presto 等引擎,以及表格式数据的管理功能。
  • 生态插件。除了默认提供 JindoSDK 支持 Hadoop,另外还支持 Flink Connector 等插件。

发布版本

JindoData 6.1.0,2023-09-28,Release Note [NEW]

JindoData 6.0.1,2023-08-18,Release Note

JindoData 6.0.0,2023-08-15,Release Note

JindoData 5.0.0,2023-07-17,Release Note

JindoData 4.6.12,2023-09-28,Release Note

JindoData 4.6.11,2023-07-24,Release Note

JindoData 4.6.10,2023-06-30,Release Note

JindoData 4.6.8,2023-05-23,Release Note

JindoData 4.6.7,2023-04-30,Release Note

JindoData 4.6.6,2023-04-02,Release Note

JindoData 4.6.5,2023-03-22,Release Note

JindoData 4.6.4,2023-02-15,Release Note

JindoData 4.6.2,2022-11-29,Release Note

JindoData 4.6.1,2022-11-14,Release Note

JindoData 4.6.0,2022-10-25,Release Note

JindoData 4.5.1,2022-09-02,Release Note

JindoData 4.5.0,2022-07-25,Release Note

JindoData 4.4.0,2022-05-30,Release Note

JindoData 4.3.0,2022-03-30,Release Note

JindoData 4.2.0,2022-01-30,Release Note

JindoData 4.1.0,2022-01-21,Release Note

JindoData 4.0.0,2022-01-07,Release Note

JindoData 3.x 版本,原阿里云 EMR SmartData

用户文档

支持阿里云 OSS 用户文档

支持阿里云 OSS-HDFS 服务(JindoFS 服务) 用户文档

支持JindoFSx 存储加速系统 用户文档

支持多云对象存储 用户文档

支持阿里云 OSS 用户文档 (3.x 版本)

JindoData 下载

JindoData 6.x 下载页面
JindoData 5.x 下载页面
JindoData 4.x 下载页面
JindoData 3.x 下载页面

用户交流群

钉钉搜索群号加入: 33413498

扫码入群:

常见问题

进入常见问题

More Repositories

1

oss-browser

OSS Browser 提供类似windows资源管理器功能。用户可以很方便的浏览文件,上传下载文件,支持断点续传等。
JavaScript
3,175
star
2

aliyun-openapi-java-sdk

Alibaba Cloud SDK for Java
Java
1,379
star
3

aliyun-oss-java-sdk

Aliyun OSS SDK for Java
Java
1,216
star
4

alibaba-cloud-sdk-go

Alibaba Cloud SDK for Go
Go
1,104
star
5

alicloud-android-demo

Java
990
star
6

aliyun-openapi-python-sdk

Alibaba Cloud SDK for Python
Python
980
star
7

aliyun-oss-php-sdk

Aliyun OSS SDK for PHP
PHP
975
star
8

aliyun-oss-go-sdk

Aliyun OSS SDK for Go
Go
951
star
9

aliyun-oss-python-sdk

Aliyun OSS SDK for Python
Python
935
star
10

darabonba

Darabonba 是一种用于 OpenAPI 的 DSL 语言,可以用来生成多语言的 SDK、Code Sample、Test Case 等代码
JavaScript
894
star
11

alibabacloud-alfa

阿里云微前端解决方案
TypeScript
845
star
12

aliyun-oss-android-sdk

Android SDK for aliyun object storage service
Java
793
star
13

aliyun-cli

Alibaba Cloud CLI
Go
770
star
14

ossfs

Export s3fs for aliyun oss.
C++
735
star
15

aliyun-openapi-php-sdk

[Abandoned] Open API SDK for PHP developers
PHP
605
star
16

terraform-provider-alicloud

Terraform AliCloud provider
Go
590
star
17

aliyun-openapi-net-sdk

Alibaba Cloud SDK for .NET
C#
535
star
18

rds_dbsync

围绕 PostgreSQL Greenplum ,实现易用的数据的互迁功能项目
C
528
star
19

openapi-sdk-php

Alibaba Cloud SDK for PHP
PHP
501
star
20

iotkit-embedded

高速镜像: https://code.aliyun.com/linkkit/c-sdk
C
492
star
21

ossutil

A user friendly command line tool to access AliCloud OSS.
Go
456
star
22

aliyun-oss-ios-sdk

iOS SDK for aliyun object storage service
Objective-C
450
star
23

aliyun-odps-python-sdk

ODPS Python SDK and data analysis framework
Python
433
star
24

alicloud-ios-demo

Demos for AMS iOS SDKs
Objective-C
431
star
25

alibabacloud-microservice-demo

An Alibaba Cloud native microservice demo powered by Apache Dubbo and Spring Cloud Alibaba
Java
379
star
26

api-gateway-demo-sign-java

aliyun api gateway request signature demo by java
Java
371
star
27

NeWCRFs

Python
365
star
28

aliyun-oss-csharp-sdk

Aliyun OSS SDK for C#
C#
360
star
29

surftrace

surftrace is a tool that allows you to surf the linux kernel
Python
332
star
30

conditional-lane-detection

Python
328
star
31

aliyun-log-jaeger

Go
294
star
32

coolbpf

C
240
star
33

tablestore-examples

Example code for aliyun tablestore.
Java
238
star
34

tablestore-timeline

TableStore-Timeline Model for Social scene
Java
236
star
35

aliyun-log-c-sdk

Aliyun LOG Producer for C/C++
C
215
star
36

openapi-sdk-php-client

Official repository of the Alibaba Cloud Client for PHP
PHP
214
star
37

aliyun-log-logback-appender

Java
186
star
38

aliyun-log-android-sdk

Java
179
star
39

openapi-core-nodejs-sdk

OpenAPI POP core SDK for Node.js
JavaScript
175
star
40

aliyun-emapreduce-datasources

Extended datasource support for Spark/Hadoop on Aliyun E-MapReduce.
Scala
168
star
41

aliyun-log-python-sdk

Use python to manage, produce and consume data with Aliyun Log Service.
Python
166
star
42

data-mapping-component

A React Component which focus on Data-Mapping & Table-Field-Mapping.(基于React的数据/表字段映射组件)
JavaScript
155
star
43

aliyun-oss-react-native

Objective-C
148
star
44

aliyun-apsaradb-hbase-demo

C++
146
star
45

django-oss-storage

Django storage backends for AliCloud OSS.
Python
144
star
46

aliyun-oss-c-sdk

Aliyun OSS SDK for C
C
144
star
47

react-visual-modeling

A DAG React Component for visualization modeling, suitable for UML, database modeling, data warehouse construction.(一个基于React的数据可视化建模的DAG图,适用于UML,数据库建模,数据仓库建设等业务)
JavaScript
138
star
48

aliyun-oss-ruby-sdk

Aliyun OSS SDK for Ruby
Ruby
138
star
49

ram-policy-editor

AliCloud RAM Policy Editor for OSS
JavaScript
136
star
50

aliyun-log-java-sdk

Java
135
star
51

serverless-aliyun-function-compute

Serverless Alibaba Cloud Function Compute Plugin – Add Alibaba Cloud Function Compute support to the Serverless Framework
JavaScript
134
star
52

alibabacloud-console-components

阿里云企业云管理平台 UI 组件库
TypeScript
133
star
53

aliyun-log-java-producer

Aliyun LOG Java Producer
Java
131
star
54

fc-nodejs-sdk

The Node.js SDK of FunctionCompute.
JavaScript
130
star
55

aliyun-cms-grafana

JavaScript
127
star
56

aliyun-odps-jdbc

JDBC Driver for ODPS
Java
125
star
57

alibabacloud-quantization-networks

alibabacloud-quantization-networks
Python
122
star
58

aliyun-emapreduce-demo

Java
121
star
59

aliyun-maxcompute-data-collectors

Java
119
star
60

alicloud-ams-demo

C#
117
star
61

alibabacloud-iot-device-sdk

alibaba cloud for iot device javascript SDK , connect with linkplatform , run at node/broswer/winxin min program /ali min program. 阿里云IoT物联网平台javascript版本sdk,可以运行在node/broswer/winxin min program /ali min program. 阿里云IoT物联网平台javascript版本sdk,可以运行在node/broswer/winxin min program /ali min program
JavaScript
110
star
62

MaxCompute-Spark

MaxCompute spark demo for building a runnable application.
Scala
106
star
63

api-gateway-nodejs-sdk

The API Gateway SDK for Node.js
JavaScript
104
star
64

cloud-design

阿里云前端组件库,由专有云&公有云前端团队共建
CSS
99
star
65

gm-jsse

开源国密通信纯 Java JSSE 实现
Java
95
star
66

aliyun-odps-console

ODPS Console Source Code.
Java
93
star
67

aliyun-openapi-cpp-sdk

Alibaba Cloud SDK for C++
C++
90
star
68

aliyun-odps-java-sdk

ODPS SDK for Java Developers
Java
89
star
69

aliyun-tablestore-nodejs-sdk

Aliyun TableStore(原OTS) SDK for Node.js
JavaScript
88
star
70

algorithm-base

让算法工程化更简单
Python
86
star
71

aliyun-log-ios-sdk

Aliyun LOG iOS SDK
Swift
84
star
72

iotx-api-demo

PHP
82
star
73

plugsched

Live upgrade Linux kernel scheduler subsystem
Python
82
star
74

DCT-Mask

Python
81
star
75

aliyun-openapi-nodejs-sdk

Alibaba Cloud SDK for Node.js
JavaScript
80
star
76

aliyun-specs

Aliyun Mobile Service CocoaPods specs.
Ruby
77
star
77

alibabacloud-console-design

阿里云管平台研发解决方案
TypeScript
77
star
78

alibabacloud-redis-training-demo

Java
76
star
79

aliyun-oss-php-sdk-laravel

A Laravel service provider for the AliCloud OSS SDK for PHP
PHP
75
star
80

aliyun-tablestore-go-sdk

TableStore SDK for Golang
Go
75
star
81

alibabacloud-sdk

Tea
75
star
82

fc-docker

Dockerfiles for local building or running function of FC
Dockerfile
74
star
83

elasticsearch-repository-oss

Java
74
star
84

dro-sfm

Python
74
star
85

packagist-mirror

Alibaba Cloud Packagist Mirror
Go
73
star
86

react-monitor-dag

A React-based operation/monitoring DAG diagram.(基于React的运维/监控DAG图)
JavaScript
69
star
87

aliyun_assist_client

Aliyun Assist Client 阿里云 云助手
Go
67
star
88

aliyun-log-php-sdk

PHP
67
star
89

alibabacloud-hologres-connectors

alibabacloud-hologres-connectors
Java
66
star
90

aliyun-tsdb-java-sdk

Aliyun TSDB SDK for Java
Java
64
star
91

aliyun-log-log4j-appender

aliyun-log-log4j-appender
Java
63
star
92

fc-java-sdk

The Java SDK of FunctionCompute.
Java
61
star
93

oss-ftp

The ftp proxy for Aliyun OSS.
Python
61
star
94

react-lineage-dag

JavaScript
61
star
95

aliyun-log-cli

Command Line Interface for Aliyun Log Service
Python
60
star
96

csb-sdk

The CSB-SDK is a client-side invocation SDK for HTTP or Web Service API opened by the CSB (Cloud Service Bus) product. It is responsible for invoking the open API and signing the request information.
Java
58
star
97

ossimport

Data migration tool
58
star
98

aliyun-log-flink-connector

flink log connector
Java
58
star
99

oss-emulator

OSS Emulator
Ruby
58
star
100

aliyun-log-dotnetcore-sdk

C#
55
star