Universal ID
URL-Safe Portability for any Ruby Object
Universal ID introduces a paradigm shift in Ruby development with powerful recursive serialization. This innovative library transforms any Ruby object into a URL-safe string, enabling efficient encoding and seamless data transfer across process boundaries. By simplifying complex serialization tasks, Universal ID enhances both the developer and end-user experience, paving the way for a wide range of use casesβfrom state preservation in web apps to inter-service communication.
It leverages both MessagePack and Brotli (a combo built for speed and best-in-class data compression). MessagePack + Brotli is up to 30% faster and within 2-5% compression rates compared to Protobuf. β
Table of Contents
- Example Use Cases
- Supported Data Types
- Settings and Prepack Options
- Advanced ActiveRecord
- ActiveRecord::Relation Support
- SignedGlobalID
- Fingerprinting (Implicit Versioning)
- Performance and Benchmarks
- Sponsors
- License
Example Use Cases
Universal ID's powerful serialization capabilities unlock a myriad of possibilities across various domains. Here are a few possibilities.
-
State Management for Web Applications: Facilitate seamless user experiences in web applications by preserving and transferring UI states, even across different sessions.
-
Data Serialization for Distributed Systems: Enable efficient communication in distributed systems by serializing complex data structures for network transmission.
-
Configuration Settings for Software Applications: Store and manage configuration settings for software applications, allowing easy transfer and versioning of settings across installations.
-
Session Continuity in Cloud Services: Ensure continuity of user sessions in cloud-based applications, enabling users to pick up their work exactly where they left off, regardless of the device or location.
-
Audit Logging for Complex Transactions: Record detailed states of complex transactions in audit logs, providing a comprehensive and reversible record of actions for compliance and analysis.
-
Machine-to-Machine Communication: Standardize data formats for machine-to-machine communication, facilitating interoperability and data exchange in IoT and other automated systems.
These use cases demonstrate the versatility and power of Universal ID in various application and business scenarios, offering solutions that enhance efficiency, user experience, and system reliability.
See More Use Cases... βΎ
Harnessing the capabilities of this advanced low-level tool in your libraries and applications might initially seem daunting. To assist you in unlocking its full potential, here are some additional suggestions to help you get started.
-
API Response Caching Cache complex API responses as serialized strings, allowing for efficient storage and quick retrieval.
-
Asset Management in Enterprises Serialize asset information, including status and location, for efficient tracking and management.
-
Audit Logging for Financial Transactions Serialize transaction states for audit trails in financial applications, providing detailed and reversible records for compliance.
-
Automated Testing of Web Applications Serialize application states to reproduce and test various scenarios automatically.
-
Backup and Restore of Application States Create snapshots of application states that can be backed up and later restored.
-
Configuration Management in DevOps Serialize configuration settings for software deployments, enabling easy versioning and rollback.
-
Content Management Systems (CMS) Serialize page or post states in CMS, enabling advanced versioning and preview functionalities.
-
Customer Support Tools Serialize user issues and their context, helping support teams to quickly understand and resolve customer problems.
-
Data Migration Between Databases Serialize entire database records for easy transfer between different database systems or formats.
-
Educational Platforms Serialize user progress and states in educational platforms, allowing students to pause and resume their learning activities.
-
E-commerce Cart Persistence Serialize shopping cart contents, enabling users to return to a filled cart even after their session expires.
-
Energy Management Systems Serialize energy usage data from various sensors for analysis and monitoring.
-
Environmental Monitoring Systems Serialize sensor data from environmental monitoring systems for analysis and historical record keeping.
-
Event Sourcing in Applications Use serialized states for event sourcing, maintaining an immutable log of state changes over time.
-
Gaming State Preservation Store game states as serialized strings, allowing players to resume games exactly where they left off.
-
Healthcare Data Exchange Securely transfer patient data between different healthcare systems while maintaining the integrity of complex data structures.
-
IoT Device State Management Serialize the state of IoT devices for efficient transmission over networks, aiding in remote monitoring and control.
-
Legal Document Management Serialize versions of legal documents, maintaining a trail of edits and changes for auditing.
-
Machine Learning Data Preparation Serialize complex data structures used in machine learning pipelines for efficient processing.
-
Microservice Communication Serialize complex objects for inter-service communication, ensuring efficient data transfer and reducing the need for complex parsing logic.
-
Real Estate Portfolio Management Serialize complex property data for portfolio management and analysis.
-
Real-time Collaboration Tools Serialize document or application states for real-time collaboration tools, ensuring consistency across different user sessions.
-
Research Data Management Serialize research data and experimental setups for ease of sharing and replication of experiments.
-
Retail Inventory Management Serialize inventory data, including details of products, for efficient management and tracking.
-
Session Continuity Across Devices Store user session data as a serialized string, enabling users to resume their session on a different device without loss of context.
-
State Management for Single Page Applications (SPAs) Serialize UI states into URL-safe strings, enabling bookmarking or sharing of specific application states.
-
Supply Chain Logistics Serialize complex logistics and shipment data, aiding in efficient tracking and management.
-
Telecommunication Network Management Serialize configurations and states of network devices for efficient management and troubleshooting.
-
Travel Itinerary Planning Serialize travel plans and itineraries, allowing users to save and share their travel details easily.
-
Version Control of Design Files Serialize design artifacts for version control in graphic design and CAD applications.
π Ready to Dive In?: All the code examples below can be tested on your local machine. Simply clone the repo and run
bin/console
to begin exploring. Don't forget to executebundle
first to ensure all dependencies are up to date. Happy coding!
Supported Data Types
Scalars
Universal ID supports most Ruby primitives.
NilClass
Complex
Date
DateTime
FalseClass
Float
Integer
Range
Rational
Regexp
String
Symbol
Time
TrueClass
You can use Universal ID for individual primitives if desired, but scalar support is really the foundation for more serious use cases. See below...
uri = URI::UID.build(:demo).to_s
#=> "uid://universal-id/iwKA1gBkZW1vAw"
uid = URI::UID.parse(uri)
#=> #<URI::UID uid://universal-id/iwKA1gBkZW1vAw>
uid.decode
#=> :demo
Composites
Composite support is where things start to get interesting. All of the composite datatypes listed below can be recursively transformed into a Universal ID.
[]
Array... βΎ
array = [1, 2, 3, [:a, :b, :c, [true]]]
uri = URI::UID.build(array).to_s
#=> "uid://universal-id/iweAlAECA5TUAGHUAGLUAGORwwM"
uid = URI::UID.parse(uri)
#=> #<URI::UID uid://universal-id/iweAlAECA5TUAGHUAGLUAGORwwM>
uid.decode
#=> [1, 2, 3, [:a, :b, :c, [true]]]
{}
Hash... βΎ
hash = {a: 1, b: 2, c: 3, array: [1, 2, 3, [:a, :b, :c, [true]]]}
uri = URI::UID.build(hash).to_s
#=> "uid://universal-id/CxKAhNQAYQHUAGIC1ABjA8cFAGFycmF5lAEC..."
uid = URI::UID.parse(uri)
#=> #<URI::UID uid://universal-id/CxKAhNQAYQHUAGIC1ABjA8cFAGFycmF5lAECA5TUAGHUAGLUAGORwwM>
uid.decode
#=> {:a=>1, :b=>2, :c=>3, :array=>[1, 2, 3, [:a, :b, :c, [true]]]}
<>
Open Struct... βΎ
ostruct = OpenStruct.new(
name: "Wireless Keyboard",
price: 49.99,
category: "Electronics",
in_stock: true
)
uri = URI::UID.build(ostruct).to_s
#=> "uid://universal-id/iyaAx0sMhNYAbmFtZbFXaXJlbGVzcyBLZXlib2FyZMcFAHByaWNly0BI_rhR64Uf1wBjYXRlZ29ye..."
uid = URI::UID.parse(uri)
#=> #<URI::UID scheme=uid, host=universal-id, payload=iyaAx0sMhNYAbmFtZbFXaXJlbGVzcyBLZXlib2FyZMcFAHByaWNly0BI_rhR64Uf1wBjYXRlZ29ye...>
uid.decode
#=> #<OpenStruct name="Wireless Keyboard", price=49.99, category="Electronics", in_stock=true>
()
Set... βΎ
set = Set.new([1, 2, 3, [:a, :b, :c, [true]]])
uri = URI::UID.build(set).to_s
#=> "uid://universal-id/iwiA2AuUAQIDlNQAYdQAYtQAY5HDAw"
uid = URI::UID.parse(uri)
#=> #<URI::UID uid://universal-id/iwiA2AuUAQIDlNQAYdQAYtQAY5HDAw>
URI::UID.parse(uri).decode
#=> #<Set: {1, 2, 3, [:a, :b, :c, [true]]}>
<>
Struct... βΎ
Book = Struct.new(:title, :author, :isbn, :published_year)
book = Book.new("The Great Gatsby", "F. Scott Fitzgerald", "9780743273565", 1925)
uri = URI::UID.build(book).to_s
#=> "uid://universal-id/G2YAoGTomv9N_4RV2oJRxRvZdC1wNJ0H3Ipu45kVcSrAxtg6Wjtogpi6GV1XXQAOAXoNR3BrCg9AQ..."
uid = URI::UID.parse(uri)
#=> #<URI::UID scheme=uid, host=universal-id, payload=G2YAoGTomv9N_4RV2oJRxRvZdC1wNJ0H3Ipu45kVcSrAxtg6Wjtogpi6GV1XXQAOAXoNR3BrCg9AQ...>
uid.decode
#=> #<struct Book title="The Great Gatsby", author="F. Scott Fitzgerald", isbn="9780743273565", published_year=1925>
Custom Types
Universal ID is extensible so you can register your own datatypes with specialized serialization rules. It couldn't be simpler. Just convert the required data to a Ruby scalar or composite value.
How to Register your own Datatype... βΎ
class UserSettings
attr_accessor :user_id, :preferences
def initialize(user_id, preferences = {})
@user_id = user_id
@preferences = preferences
end
end
UniversalID::MessagePackFactory.register(
type: UserSettings,
packer: ->(user_preferences, packer) do
packer.write user_preferences.user_id
packer.write user_preferences.preferences
end,
unpacker: ->(unpacker) do
user_id = unpacker.read
preferences = unpacker.read
UserSettings.new user_id, preferences
end
)
settings = UserSettings.new(1,
theme: "dark",
notifications: "email",
language: "en",
layout: "grid",
privacy: "private"
)
uri = URI::UID.build(settings).to_s
#=> "uid://universal-id/G1QAQAT-bfcGW1QOgadJwJF06yL8gDnGgfs1Xdti20TDDvG5STPqzbYcQ6TBqVKhdZ39CdQZUwEGe..."
uid = URI::UID.parse(uri)
#=> #<URI::UID uid://universal-id/G1QAQAT-bfcGW1QOgadJwJF06yL8gDnGgfs1Xdti20TDDvG5STPqzbYcQ6TBqVKhdZ39CdQZUwEGe..."
uid.decode
=> #<UserSettings:0x0000000139157dd8 @preferences={:theme=>"dark", :notifications=>"email", :language=>"en", :layout=>"grid", :privacy=>"private"}, @user_id=1>
Contributed Types
Universal ID is designed to be highly extensible, allowing for third-party contributions to enhance its capabilities. These contributions can introduce support for additional data types, further broadening the scope of Universal IDβs utility. The following are some notable contrib extensions:
-
ActiveRecord::Base: Integrates Universal ID with ActiveRecord base models, enabling intelligent serialization of database records.
-
ActiveRecord::Relation: Supports the serialization of ActiveRecord relations, making it possible to encode complex query structures.
-
ActiveSupport::TimeWithZone: Adds the ability to serialize ActiveSupport's TimeWithZone objects.
-
GlobalID: Extends support to include GlobalIDs.
-
SignedGlobalID: Extends support to include SignedGlobalIDs.
Requiring Contributed Types
To utilize the contributed types, you must explicitly require them in your application. This ensures the extensions are loaded and available for use. Here is an example illustrating how to include contributed types:
# load contrib types
require "universal_id/contrib/active_record"
require "universal_id/contrib/active_support"
require "universal_id/contrib/global_id"
require "universal_id/contrib/signed_global_id"
# or simply
require "universal_id/contrib/rails"
π‘ Implicit Contribs: Whenever the
Rails
constant is defined, the related contribs are auto-loaded.
π‘ Broad Compatibility: Universal ID has built-in support for ActiveRecord, yet it maintains independence from Rails-specific dependencies. This versatile design enables integration into any Ruby project.
Why Universal ID with ActiveRecord?
While ActiveRecord already supports GlobalID, a robust library for serializing individual ActiveRecord models, Universal ID extends this functionality to cover a wider range of use cases. Here are a few reasons you may want to consider Universal ID.
-
Support for New Records: Unlike GlobalID, Universal ID can serialize models that haven't been saved to the database yet.
-
Capturing Unsaved Changes: It can serialize ActiveRecord models with unsaved changes, ensuring that even transient states are captured.
-
Association Handling: Universal ID goes beyond single models. It can serialize associated records, including those with unsaved changes, creating a comprehensive snapshot of complex object states.
-
Cloning Existing Records: Need to make a copy of a record, including its associations? Universal ID handles this effortlessly, making it ideal for duplicating complex datasets.
-
Granular Data Control: With Universal ID, you gain explicit control over the serialization process. You can precisely choose which columns to include or exclude, allowing for tailored, optimized payloads that fit your specific needs.
-
Efficient Query Serialization: Universal ID extends its capabilities to ActiveRecord relations, enabling the serialization of complex queries and scopes. This feature allows for seamless sharing of query logic between processes, ensuring consistency and reducing redundancy in data handling tasks.
In summary, while GlobalID excels in its specific use case, Universal ID offers extended capabilities, particularly useful in scenarios involving unsaved records, complex associations, and data cloning.
How to Convert Records to UIDs... βΎ
ActiveRecord::Base.establish_connection(adapter: "sqlite3", database: ":memory:")
ActiveRecord::Schema.define do
create_table :campaigns do |t|
t.column :name, :string
t.timestamps
end
end
class Campaign < ApplicationRecord
end
# ---
campaign = Campaign.create(name: "Marketing Campaign")
uri = URI::UID.build(campaign).to_s
#=> "uid://universal-id/CwiAxw4EqENhbXBhaWdugaJpZAMD"
uid = URI::UID.parse(uri)
#=> #<URI::UID uid://universal-id/CwiAxw4EqENhbXBhaWdugaJpZAMD>
URI::UID.parse(uri).decode
##<Campaign:0x000000011cc67da8 id: 1, name: "Marketing Campaign", ...>
Settings and Prepack Options
Universal ID supports a small but powerful set of configuration options for transforming objects before being handed off to MessagePack for serialization.
Prepacking gives you explicit control over what data to include in the Universal ID.
View All Settings and Prepack Options... βΎ
prepack:
# ..........................................................................................................
# A list of attributes to exclude (for objects like Hash, OpenStruct, Struct, etc.)
# Takes prescedence over the`include` list
exclude: []
# ..........................................................................................................
# A list of attributes to include (for objects like Hash, OpenStruct, Struct, etc.)
include: []
# ..........................................................................................................
# Whether or not to omit blank values when packing (nil, {}, [], "", etc.)
include_blank: true
# ==========================================================================================================
# Database records
database:
# ......................................................................................................
# Whether or not to include primary/foreign keys
# Setting this to `false` can be used to make a copy of an existing record
include_keys: true
# ......................................................................................................
# Whether or not to include date/time timestamps (created_at, updated_at, etc.)
# Setting this to `false` can be used to make a copy of an existing record
include_timestamps: true
# ......................................................................................................
# Whether or not to include unsaved changes
# Assign to `true` when packing new records
include_unsaved_changes: false
# ......................................................................................................
# Whether or not to include loaded in-memory descendants (i.e. child associations)
include_descendants: false
# ......................................................................................................
# The max depth (number) of loaded in-memory descendants to include when `include_descendants == true`
# For example, a value of (3) would include the following:
# Parent > Child > Grandchild
descendant_depth: 0
Prepack options can be applied when creating a Universal ID and can be passed in structured or flat format.
How to Apply Prepack Options when Creating UIDs... βΎ
person = {
full_name: "Jane Doe",
email: "[email protected]",
birthdate: "1980-05-15",
phone_number: "555-6789",
ssn: "123-45-6789",
children: [
{
full_name: "Alice Doe",
email: "[email protected]",
birthdate: nil,
phone_number: "555-1234",
ssn: "987-65-4321"
},
{
full_name: "Bob Doe",
email: "[email protected]",
birthdate: "2008-11-21",
phone_number: nil,
ssn: "456-12-1234"
}
]
}
uid = URI::UID.build(person, include_blank: false, exclude: [:phone_number, :ssn])
uid.decode
# Note that the decoded payload is smaller due to the prepack options
# Also note that the options were applied recursively
{
full_name: "Jane Doe",
email: "[email protected]",
birthdate: "1980-05-15",
children: [
{
full_name: "Alice Doe",
email: "[email protected]"
},
{
full_name: "Bob Doe",
email: "[email protected]",
birthdate: "2008-11-21"
}
]
}
It's also possible to register frequently used options as reusable settings to further simplify creating UIDs.
How to Register Prepack Options as Preconfigured Settings... βΎ
# app/config/unsaved.yml
prepack:
include_blank: false
database:
include_unsaved_changes: true
include_timestamps: false
UniversalID::Settings.register :unsaved, File.expand_path("app/config/unsaved.yml", __dir__)
URI::UID.build @record, UniversalID::Settings[:small_record]
Advanced ActiveRecord
Universal ID includes some advanced capabilities when used with ActiveRecord.
-
Include loaded associations Universal ID supports including
loaded
associations when a model is transformed into a UID. Note that associations must beloaded?
to be considered candidates for inclusion. There are multiple ways to achieve this, so be sure to read up on associations. -
Include unsaved changes Universal ID supports capturing unsaved change, for both new and persisted records, when a model is transformed into a UID. This allows you to marshal complex unsaved data that can be restored at a later time. This feature supports several use cases, like allowing users to pause their work and resume at any point in the future without the need to store partial records in your database. And, because UIDs are web safe, you can hold this data in URLs, browser Cookies, Local/SessionStorage, etc.
-
Exclude keys to make copies of existing records Universal ID supports making copies of individual records or entire collections by opt'ing to exclude keys when transorming to UID. This allows you to make data sharable. Consider a sencario with complex infrastructure (db sharding, etc.). You can leverage Universal ID to move entire subsets of data across physical data stores.
First, let's establish the schema structure and data we'll be working with. We'll limit ourselves to 3 tables here, but Universal ID can support much more complex data models.
- Campaign
- Attachment
We'll use 1 campaign with 3 emails that have 2 attachments each.
Setup the Schema... βΎ
ActiveRecord::Base.establish_connection(adapter: "sqlite3", database: ":memory:")
ActiveRecord::Schema.define do
create_table :campaigns do |t|
t.column :name, :string
t.column :description, :text
t.column :trigger, :string
t.timestamps
end
create_table :emails do |t|
t.column :campaign_id, :integer
t.column :subject, :string
t.column :body, :text
t.column :wait, :integer
t.timestamps
end
create_table :attachments do |t|
t.column :email_id, :integer
t.column :file_name, :string
t.column :content_type, :string
t.column :file_size, :integer
t.column :file_data, :binary
t.timestamps
end
end
Setup the Models... βΎ
class Campaign < ApplicationRecord
has_many :emails, dependent: :destroy
end
class Email < ApplicationRecord
belongs_to :campaign
has_many :attachments, dependent: :destroy
end
class Attachment < ApplicationRecord
belongs_to :email
end
Setup the Model Instances... βΎ
campaign = Campaign.new(
name: "Summer Sale Campaign",
description: "A campaign for the summer sale, targeting our loyal customers.",
trigger: "SummerStart"
)
# NOTE: Assigning campaign.emails via `=` to ensure ActiveRecord flags the association as `loaded`
campaign.emails = 3.times.map do |i|
email = campaign.emails.build(
subject: "Summer Sale Special Offer #{i + 1}",
body: "Dear Customer, check out our exclusive summer sale offers! #{i + 1}",
wait: rand(1..14)
)
# NOTE: Assigning email.attachments via `=` to ensure ActiveRecord flags the association as `loaded`
email.tap do |e|
e.attachments = 2.times.map do |j|
data = SecureRandom.random_bytes(rand(500..1500))
e.attachments.build(
file_name: "summer_sale_#{i + 1}_attachment_#{j + 1}.pdf",
content_type: "application/pdf",
file_size: data.size,
file_data: data
)
end
end
end
# demonstrate that we have new unsaved records
#campaign
campaign.new_record? # true
campaign.changed? # true
# emails
campaign.emails.each do |email|
email.new_record? # true
email.changed? # true
email.attachments.each do |attachment|
attachment.new_record? # true
attachment.changed? # true
end
end
Now let's look at how to leverage Universal ID with ActiveRecord.
How to Include Unsaved Changes for New Records... βΎ
# prepack options
options = {
include_unsaved_changes: true,
include_descendants: true,
descendant_depth: 2
}
# NOTE: The campaign model instance was setup earlier in the "Model Instances" section above
campaign.new_record? # true
campaign.changes
# {
# "name"=>[nil, "Summer Sale Campaign"],
# "description"=>[nil, "A campaign for the summer sale, targeting our loyal customers."],
# "trigger"=>[nil, "SummerStart"]
# }
campaign.emails.each do |email|
email.new_record? # true
email.changes
# {
# "subject"=>[nil, "Summer Sale Special Offer ..."],
# "body"=>[nil, "Dear Customer, check out our exclusive summer sale offers! ..."],
# "wait"=>[nil, ...]
# }
email.attachments.each do |attachment|
attachment.new_record? # true
attachment.changes
# {
# "file_name"=>[nil, "summer_sale_..._attachment_....pdf"],
# "content_type"=>[nil, "application/pdf"],
# "file_size"=>[nil, ...],
# "file_data"=>[nil, "..."]
# }
end
end
encoded = URI::UID.build(campaign, options).to_s
restored = URI::UID.parse(encoded).decode
restored.new_record? # true
restored.changes
# {
# "name"=>[nil, "Summer Sale Campaign"],
# "description"=>[nil, "A campaign for the summer sale, targeting our loyal customers."],
# "trigger"=>[nil, "SummerStart"]
# }
restored.emails.each do |email|
email.new_record? # true
email.changes
# {
# "subject"=>[nil, "Summer Sale Special Offer ..."],
# "body"=>[nil, "Dear Customer, check out our exclusive summer sale offers! ..."],
# "wait"=>[nil, ...]
# }
email.attachments.each do |attachment|
attachment.new_record? # true
attachment.changes
# {
# "file_name"=>[nil, "summer_sale_..._attachment_....pdf"],
# "content_type"=>[nil, "application/pdf"],
# "file_size"=>[nil, ...],
# "file_data"=>[nil, "..."]
# }
end
end
How to Include Unsaved Changes for Persisted Records... βΎ
# NOTE: The campaign model instance was setup earlier in the "Model Instances" section above
# persist the model and its associations
campaign.save!
# make some unsaved changes to the records
campaign.name = "Changed Name #{SecureRandom.hex}"
campaign.emails.each do |email|
email.subject = "Changed Subject #{SecureRandom.hex}"
email.attachments.each do |attachment|
attachment.file_name = "changed_file_name#{SecureRandom.hex}.pdf"
end
end
campaign.persisted? # true
campaign.changes
# {"name"=>["Summer Sale Campaign", "Changed Name ..."]}
campaign.emails.each do |email|
email.persisted? # true
email.changes
# {"subject"=>["Summer Sale Special Offer 1", "Changed Subject ..."]}
email.attachments.each do |attachment|
attachment.persisted? # true
attachment.changes
# {"file_name"=>["summer_sale_..._attachment_....pdf", "changed_file_name....pdf"]}
end
end
# prepack options
options = {
include_unsaved_changes: true,
include_descendants: true,
descendant_depth: 2
}
encoded = URI::UID.build(campaign, options).to_s
restored = URI::UID.parse(encoded).decode
restored.persisted? # true
restored.changes
# {"name"=>["Summer Sale Campaign", "Changed Name ..."]}
restored.emails.each do |email|
email.persisted? # true
email.changes
# {"subject"=>["Summer Sale Special Offer 1", "Changed Subject ..."]}
email.attachments.each do |attachment|
attachment.persisted? # true
attachment.changes
# {"file_name"=>["summer_sale_..._attachment_....pdf", "changed_file_name....pdf"]}
end
end
How to Copy Persisted Records... βΎ
# NOTE: The campaign model instance was setup earlier in the "Model Instances" section above
# persist the model and its associations
campaign.save!
options = {
include_keys: false,
include_timestamps: false,
include_unsaved_changes: true,
include_descendants: true,
descendant_depth: 2
}
encoded = URI::UID.build(campaign, options).to_s
copy = URI::UID.parse(encoded).decode
campaign.persisted? # false
copy.new_record? # true
copy.save!
copy == campaign # false
campaign.emails.each do |email|
copy_email_ids = copy.emails.map(&:id)
campaign_email_ids = campaign.emails.map(&:id)
(copy_email_ids && campaign_email_ids).any? # false
copy_attachment_ids = copy.emails.map(&:attachments).flatten.map(&:id)
campaign_attachment_ids = campaign.emails.map(&:attachments).flatten.map(&:id)
(copy_attachment_ids & campaign_attachment_ids).any? # false
end
ActiveRecord::Relation Support
Universal ID seamlessly handles the serialization of ActiveRecord relations and scopes, striking the perfect balance between efficiency and functionality. It paves the way for easy, optimized, and effective sharing of database queries. This capability transforms query management, allowing developers to encapsulate complex query structures into a reliable, portable, and reusable format that ensures query consistency across different parts of the application.
π‘ Optimized Payloads: When handling
ActiveRecord::Relations
, Universal ID intelligently clears cached data within the relation before encoding. This approach minimizes payload size, ensuring efficient data transfer without sacrificing the integrity of the original query logic.
How to work with ActiveRecord::Relations... βΎ
# Assuming we have multiple campaigns already stored in the database
relation = Campaign.joins(:emails).where("emails.subject LIKE ?", "Flash Sale%")
# force load the relation
relation.load
relation.loaded? # true
encoded = URI::UID.build(relation).to_s
decoded = URI::UID.parse(encoded).decode
decoded.is_a? ActiveRecord::Relation # true
decoded.loaded? # false
decoded == relation # true
decoded.size == relation.size # true
decoded.to_a == relation.to_a # true
SignedGlobalID
Features like signing
(to prevent tampering), purpose
, and expiration
are provided by SignedGlobalIDs.
These features (and more) will eventually be added to UniversalID, but until then...
simply convert your UniversalID to a SignedGlobalID to add these features to any Universal ID.
How to Convert a UID to/from a SignedGlobalID... βΎ
product = {
name: "Wireless Bluetooth Headphones",
price: 79.99,
category: "Electronics"
}
uid = URI::UID.build(product)
#=> #<URI::UID scheme="uid", host="universal-id", path="/G0sAgBypU587HsjkLpEnGHiaWfPQEyiiuH6j...">
sgid = uid.to_sgid_param(for: "cart-123", expires_in: 1.hour)
#=> "eyJfcmFpbHMiOnsibWVzc2FnZSI6IkJBaEpJZ0d4WjJsa09pOHZkVzVwZG1WeWMyRnNMV2xrTDFWU1NUbzZWVWxFT2pwSGJHOWlZV3hKUkZKbFkyOXlaQzlITUhO..."
URI::UID.from_sgid(sgid, for: "cart-123").decode
#=> {
# name: "Wireless Bluetooth Headphones",
# price: 79.99,
# category: "Electronics"
# }
# mismatched purpose returns nil... as expected
URI::UID.from_sgid(sgid, for: "mismatch")
#=> nil
Fingerprinting (Implicit Versioning)
Fingerprinting adds an extra layer of intelligence to the serialization process. UIDs automatically include a "fingerprint" for each serialized object based on the target object's class and its modification time (mtime).
Fingerprints are comprised of the following components:
Class (Class)
- The encoded object's classTimestamp (Time)
- The mtime (UTC) of the file that defined the object's class
π‘ Modification Timestamp: The
mtime
is detected and captured the moment a UID is created.
Fingerprints allow developers to manage different versions of serialized data effectively...without the need for custom versioning. Whenever the class definition changes, the mtime updates, resulting in a different fingerprint. This is especially useful in scenarios where the data format evolves over time, such as in long-lived applications.
How to Use Fingerprinting... βΎ
- Build a UID using a custom handler (optional Ruby block). This allows you to take control of the encoding process.
# NOTE: The campaign model instance was setup earlier in the "Model Instances" section above
campaign.save!
# the uid build target (campaign in this case)
# |
# | encoding options (whatever was passed to URI::UID.build or {})
# | |
uid = URI::UID.build(campaign) do |record, options|
# NOTE: this feature is compatible with other tools like kiba, amoeba, etc.
# that can be leveraged to generate the data to be encoded
data = { id: record.id, demo: true }
URI::UID.encode data, options.merge(include: %w[id demo]) # block returns the encoded payload
end
- Decode the UID using a custom handler (optional Ruby block). This allows you to take control of the decoding process.
# fingerprint components
# ____|______
# the decoded payload from above | |
# | | |
decoded = URI::UID.parse(uid.to_s).decode do |data, klass, timestamp|
# NOTE: this feature is compatible with other tools like kiba, amoeba, etc.
# that can be leveraged to translate the decoded data into the correct format
record = klass.find_by(id: data[:id])
record.instance_variable_set(:@demo, data[:demo])
case Time.parse(timestamp)
when 3.months.ago..Time.now
# current data format
# return the record as-is
record
when 1.year.ago..3.months.ago
# outdated data format
# apply an ETL process to bring the outdated data current
# then return the modified record
record
end
end
Fingerprinting allows for seamless handling of different data versions and formats, so you can maintain consistency and reliability in applications dealing with serialized data over time.
π‘ Optional Usage: While fingerpint creation is automatic and implicit, using it is optional... ready whenever you want more control.
Performance and Benchmarks
View Benchmarks... βΎ
Benchmarks can be performed by cloning the project and running bin/bench
.
The run below was performed on the following hardware.
Model Name: MacBook Air
Model Identifier: MacBookAir10,1
Chip: Apple M1
Total Number of Cores: 8 (4 performance and 4 efficiency)
Memory: 16 GB
Benchmarking with the following ActiveRecord/Hash data...
==================================================================================================
{
"id" => 1,
"name" => "Production",
"description" => "RISC is good Well then get your shit together. Get it all together and put it in a backpack, all your shit, so it's together. ...and if you gotta take it somewhere, take it somewhere ya know? Take it to the shit store and sell it, or put it in a shit museum. I don't care what you do, you just gotta get it together... Get your shit together. Mark it zero! What you do not smell is called Iocane Power. You wanna hear a lie? ... I...think you're great. You're my best friend.",
"trigger" => "Political Organization enhance web-enabled architectures",
"created_at" => "2023-11-11T01:28:46.657Z",
"updated_at" => "2023-11-11T01:28:46.657Z",
"emails" => [
[0] {
"id" => 1,
"campaign_id" => 1,
"subject" => "drive synergistic web-readiness",
"body" => "But first things first. To the death! I feel like all my kids grew up, and then they married each other. It's every parents' dream. Koona t'chuta Solo? (Going somewhere Solo?)",
"wait" => nil,
"created_at" => "2023-11-11T01:28:46.661Z",
"updated_at" => "2023-11-11T01:28:46.675Z",
"attachments" => [
[0] {
"id" => 1,
"email_id" => 1,
"file_name" => "Schneider and Sons",
"content_type" => "Enterprise-wide 4th generation complexity",
"file_size" => nil,
"file_data" => nil,
"created_at" => "2023-11-11T01:28:46.664Z",
"updated_at" => "2023-11-11T01:28:46.670Z"
},
[1] {
"id" => 2,
"email_id" => 1,
"file_name" => "Devolved solution-oriented circuit",
"content_type" => "revolutionize magnetic bandwidth Intelligent Paper Gloves",
"file_size" => nil,
"file_data" => nil,
"created_at" => "2023-11-11T01:28:46.664Z",
"updated_at" => "2023-11-11T01:28:46.670Z"
}
]
},
[1] {
"id" => 2,
"campaign_id" => 1,
"subject" => "Marketing",
"body" => "I'll explain and I'll use small words so that you'll be sure to understand, you warthog faced buffoon. Well then get your shit together. Get it all together and put it in a backpack, all your shit, so it's together. ...and if you gotta take it somewhere, take it somewhere ya know? Take it to the shit store and sell it, or put it in a shit museum. I don't care what you do, you just gotta get it together... Get your shit together. I am running away from my responsibilities. And it feels good.",
"wait" => nil,
"created_at" => "2023-11-11T01:28:46.671Z",
"updated_at" => "2023-11-11T01:28:46.675Z",
"attachments" => [
[0] {
"id" => 3,
"email_id" => 2,
"file_name" => "Weber-Schulist benchmark open-source applications",
"content_type" => "Enormous Linen Shoes synthesize customized e-services",
"file_size" => nil,
"file_data" => nil,
"created_at" => "2023-11-11T01:28:46.672Z",
"updated_at" => "2023-11-11T01:28:46.672Z"
},
[1] {
"id" => 4,
"email_id" => 2,
"file_name" => "thought leadership",
"content_type" => "Business Development Enhanced logistical collaboration",
"file_size" => nil,
"file_data" => nil,
"created_at" => "2023-11-11T01:28:46.672Z",
"updated_at" => "2023-11-11T01:28:46.672Z"
}
]
},
[2] {
"id" => 3,
"campaign_id" => 1,
"subject" => "Mediocre Aluminum Car",
"body" => "Donβt even trip dawg. Stay away from my special lady friend, man.",
"wait" => nil,
"created_at" => "2023-11-11T01:28:46.672Z",
"updated_at" => "2023-11-11T01:28:46.675Z",
"attachments" => [
[0] {
"id" => 5,
"email_id" => 3,
"file_name" => "Import and Export",
"content_type" => "Heavy Duty Paper Bench Project Management",
"file_size" => nil,
"file_data" => nil,
"created_at" => "2023-11-11T01:28:46.673Z",
"updated_at" => "2023-11-11T01:28:46.674Z"
},
[1] {
"id" => 6,
"email_id" => 3,
"file_name" => "synthesize ubiquitous architectures Corporate Communications",
"content_type" => "Durable Rubber Watch",
"file_size" => nil,
"file_data" => nil,
"created_at" => "2023-11-11T01:28:46.674Z",
"updated_at" => "2023-11-11T01:28:46.674Z"
}
]
}
]
}
==================================================================================================
Benchmarking 5000 iterations
==================================================================================================
user system total real
URI::UID.build Hash 14.770667 0.102535 14.873202 ( 14.898856)
Average 0.002954 0.000021 0.002975 ( 0.002980)
..................................................................................................
user system total real
URI::UID.build Hash, include_blank: false 13.821420 0.066910 13.888330 ( 13.892066)
Average 0.002764 0.000013 0.002778 ( 0.002778)
..................................................................................................
user system total real
URI::UID.parse HASH/UID 0.075566 0.000411 0.075977 ( 0.076035)
Average 0.000015 0.000000 0.000015 ( 0.000015)
..................................................................................................
user system total real
URI::UID.decode HASH/UID 0.111007 0.003572 0.114579 ( 0.114587)
Average 0.000022 0.000001 0.000023 ( 0.000023)
..................................................................................................
user system total real
URI::UID.build ActiveRecord 0.984594 0.010059 0.994653 ( 0.994662)
Average 0.000197 0.000002 0.000199 ( 0.000199)
..................................................................................................
user system total real
URI::UID.build ActiveRecord, exclude_blank 0.953653 0.006692 0.960345 ( 0.960765)
Average 0.000191 0.000001 0.000192 ( 0.000192)
..................................................................................................
user system total real
URI::UID.build ActiveRecord, include_descendants 44.958468 0.170125 45.128593 ( 45.176116)
Average 0.008992 0.000034 0.009026 ( 0.009035)
..................................................................................................
user system total real
URI::UID.parse ActiveRecord/UID 0.119030 0.000319 0.119349 ( 0.119525)
Average 0.000024 0.000000 0.000024 ( 0.000024)
..................................................................................................
user system total real
URI::UID.decode HASH/UID 5.198092 0.024652 5.222744 ( 5.282794)
Average 0.001040 0.000005 0.001045 ( 0.001057)
..................................................................................................
user system total real
UID > GID > UID.decode include_descendants 55.612061 0.398193 56.010254 ( 57.372350)
Average 0.011122 0.000080 0.011202 ( 0.011474)
..................................................................................................
user system total real
UID > SGID > UID.decode include_descendants 55.406590 0.260552 55.667142 ( 56.432082)
Average 0.011081 0.000052 0.011133 ( 0.011286)
..................................................................................................
Sponsors
Proudly sponsored by
License
The gem is available as open source under the terms of the MIT License.