Petra is a proof-of-concept for persisted transactions in Ruby with (hopefully) full ACI(D) properties.
Please note that this was created during my master's thesis in 2016 and hasn't been extended a lot since then except for a few coding style fixes. I would write a lot of stuff differently today, but the main concept is still interesting enough.
It allows starting a transaction without committing it and resuming it at a later time, even in another process - given the used objects provide identifiers other than object_id
.
It should work with every Ruby object and can be extended to work with web frameworks like Ruby-on-Rails as well (a POC of RoR integration can be found at stex/petra-rails).
This README only covers parts of what petra
has to offer. Feel free to dive into the code, everything should be commented accordingly.
Let's take a look at how petra
is used:
class SimpleUser
attr_accessor :first_name, :last_name
def name
"#{first_name} #{last_name}"
end
# ... configuration, see below
end
user = SimpleUser.petra.new('John', 'Doe')
# Start a new transaction and start changing attributes
Petra.transaction(identifier: 'tr1') do
user.first_name = 'Foo'
end
# No changes outside the transaction yet...
puts user.name #=> 'John Doe'
# Continue the same transaction
Petra.transaction(identifier: 'tr1') do
puts user.name #=> 'Foo Doe'
user.last_name = 'Bar'
end
# Another transaction changes a value already changed in 'tr1'
Petra.transaction do
user.first_name = 'Moo'
Petra.commit!
end
puts user.name #=> 'Moo Doe'
# Try to commit our first transaction
Petra.transaction(identifier: 'tr1') do
puts user.name
Petra.commit!
rescue Petra::WriteClashError => e
# => "The attribute `first_name` has been changed externally and in the transaction. (Petra::WriteClashError)"
# Let's use our value and go on with committing the transaction
e.use_ours!
e.continue!
end
# The actual object is updated with the values from tr1
puts user.name #=> 'Foo Bar'
We just used a simple Ruby object inside a transaction which was even split into multiple sections!
(The full example can be found at examples/showcase.rb
)
Simply add the following line to your gemfile:
gem 'petra_core', require: 'petra'
Unfortunately, the gem name petra
is already taken and petra-core
would express that this gem is extending it, so
I went for an underscore for now. It's hard finding nice-sounding gem names which are not yet taken nowadays :/
Whenver you call Petra.transaction
, a transaction section is started. If you pass in an identifier and a matching transaction already exists, it will be resumed instead.
# Starting a new transaction with an auto-generated identifier
tr_id = Petra.transaction {}
# Resuming the transaction
Petra.transaction(identifier: tr_id) {}
Although petra
is seemingly able to use every Ruby object inside a transaction, it does not patch these objects in any way by e.g. overriding their getters and setters. Instead, a transparent proxy is used:
# Normal instance of SimpleUser
user = SimpleUser.new
# ObjectProxy, can now be used inside and outside of transactions
user = SimpleUser.petra.new # or: user = SimpleUser.new.petra
In its current version, petra
has to be told about the meaning of the different methods of a class to be used inside a transaction.
This decision was made as there are no strict conventions regarding method names in Ruby (e.g. getX
/setX
in Java).
petra
knows about 5 different kinds of methods:
- Attribute Readers which retrieve a current attribute value
- Attribute Writers which set a new attribute value
- Dynamic Attribute Readers which a composite methods like
name
(not an actual attribute, but use attributes interally) - Persistence Methods which save changes made to the object (think of
ActiveRecord::Base#save
) - Destruction Methods which remove the object
Let's create a configuration for SimpleUser
:
Petra.configure do
configure_class SimpleUser do
# Tell petra about our available attribute readers
attribute_reader? do |method_name|
%w[first_name last_name].include?(method_name.to_s)
end
# Do the same for attribute writers
attribute_writer? do |method_name|
%w[first_name= last_name=].include?(method_name.to_s)
# also possible here: `method_name.last == '='`
end
# Define which methods are used to persist instances of SimpleUser
persistence_method? do |method_name|
%w[first_name= last_name=].include?(method_name.to_s)
end
# `name` uses attributes internally
dynamic_attribute_reader? do |method_name|
%[name].include?(method_name.to_s)
end
end
end
As you may have noticed, we used our attribute_writer
s twice in this configuration: Once as actual attribute writers and once as persistence method. This was done to keep the example above as small as possible.
The same could have been achieved by setting up a no-op method and configuring it accordingly:
# SimpleUser
def save; end
# Configuration
persistence_method { |method_name| %w[save].include?(method_name.to_s) }
# Usage
Petra.transaction do
user.first_name = 'Foo'
user.save
end
In this case, not calling save
inside the transaction would have lead to the loss of everything we did inside the transaction section.
Transactions can be committed by calling Petra.commit!
inside a Petra.transaction
block.
It will leave the transaction block afterwards and not execute anything left in it:
Petra.transaction do
Petra.commit!
puts 'I will never be shown!'
end
A rollback can be triggered by either raising Petra::Rollback
or simply any other uncaught StandardError
. The difference is that Petra::Rollback
will be swallowed by the transaction processing (like ActiveRecord::Rollback
does), while any other error will be re-raised.
Triggering a rollback will undo all changes made in the current section of the transaction. All previous sections are not affected.
Petra.transaction(identifier: 'tr1') do
user.first_name = 'Foo'
end
Petra.transaction(identifier: 'tr1') do
user.last_name = 'Bar'
fail Petra::Rollback
end
In this example, only the change to user#last_name
is lost.
A reset can be triggered by raising Petra::Reset
. It works like a rollback, but will clear the whole transaction.
Petra.transaction(identifier: 'tr1') do
user.first_name = 'Foo'
end
Petra.transaction(identifier: 'tr1') do
user.last_name = 'Bar'
fail Petra::Reset
end
Here, all changes to user
are lost.
A retry means that the current transaction block should be retried again after a rollback.
Petra.transaction(identifier: 'tr1') do
user.last_name = 'Bar'
fail Petra::Retry if some_condition
end
As the transaction is working in isolation on its own data set, it might happen that the original objects outside the transaction are changed in the meantime, e.g. by another transaction's commit:
Petra.transaction(identifier: 'tr1') do
user.first_name = 'Foo'
end
Petra.transaction(identifier: 'tr2') do
user.first_name = 'Moo'
Petra.commit!
end
Petra.transaction(identifier: 'tr1') do
# we don't know about the external change here and would
# possibly override it
end
petra
reacts to these external changes and raises a corresponding exception. This exception allows the developer to solve the conflicts based on his current context.
The exception is thrown either when the attribute is used again or during the commit phase. Not handling any of these exception yourself will result in a transaction reset.
Each error described below shares a few common methods to control the further transaction flow:
Petra.transaction(identifier: 'tr1') do
begin
...
rescue Petra::ValueComparisionError => e # Superclass of ReadIntegrityError and WriteClashError
e.object #=> the object which was changed externally
e.attribute #=> the name of the changed attribute
e.external_value #=> the new external value
e.retry! # Runs the current transaction block again
e.rollback! # Dismisses all changes in the current section, continues after transaction block
e.reset! # Resets the whole transaction, continues after transaction block
e.continue! # Continues with executing the current transaction block
end
end
Please note that in most cases calling rollback!
, retry!
or continue!
without any other exception specific method will result in the same error again the next time.
A ReadIntegrityError
is thrown if one transaction reads an attribute value which is then changed externally:
Petra.transaction(identifier: 'tr1') do
user.last_name = 'the first' if user.first_name = 'Karl'
end
user.first_name = 'Olaf'
Petra.transaction(identifier: 'tr1') do
user.first_name
#=> Petra::ReadIntegrityError: The attribute `first_name` has been changed externally.
end
When triggering a ReadIntegrityError
, you can choose to acknowledge/ignore the external change. Doing so will suppress further errors as long as the external value does not change again.
begin
...
rescue Petra::ReadIntegrityError => e
e.last_read_value #=> the value we got when last reading the attribute
e.ignore!(update_value: true) # we acknowledge the external change and use the new value in our transaction from now on
e.ignore!(update_value: false) # we keep our old value and simply ignore the external change.
e.retry!
end
A WriteClashError
is thrown whenever an attribute we changed inside one of our transaction sections was also changed externally:
Petra.transaction(identifier: 'tr1') do
user.first_name = 'Foo'
end
user.first_name = 'Moo'
Petra.transaction(identifier: 'tr1') do
user.first_name
#=> Petra:WriteClashError: The attribute `first_name` has been changed externally and in the transaction.
end
As both sides changed the attribute value, we have to decided which one to use further in most cases (or completely reset the transaction):
begin
...
rescue Petra::WriteClashError => e
e.our_value #=> the value we set the attribute to
e.their_value #=> the new external value
e.use_theirs! # undo every change we made to the attribute in this transaction
e.use_ours! # Ignore the external change, use our value
e.retry!
end
As mentioned above, petra
allows the developer to jump back into the transaction after an error was resolved.
This is done by using Ruby's Continuation which basically saves a copy of the stack at the time the exception happened. This copy can then be restored if the developer decides to continue the execution.
I'd personally keep everything regarding continuations far away from production code, but they are a very interesting concept (which will most likely be removed with Ruby 3.0 :/ ). examples/continuation_error.rb
shows one of the drawbacks which could lead to a long time of debugging.
begin
simple_user.first_name = 'Foo'
simple_user.save
rescue Petra::WriteClashError => e
e.use_ours!
# Jumps back to `simple_user.save` without a retry
e.continue!
end
Petra.configure do
persistence_adapter :file
persistence_adapter.storage_directory = '/tmp/petra'
end
Specifies the persistence adapter and its possible options.
Petra only includes a file system based adapter by default.
Petra.configure do
instantly_fail_on_read_integrity_errors false
end
petra
can be set to optimistic transaction handling. This means, that a transaction is only checked
for possible external changes during the commit phase.
By default, a corresponding error is thrown directly when the attribute is accessed again within the transaction.
Petra.configure do
log_level :debug | :info | :warn | :error
end
Specifies the log level petry
should use.
:debug
- Information about all methods called on an object proxy and their results
- Attribute reads and changes
- Acquired and released locks
- The creation of transaction log entries
:info
- Starting and persisting a transaction
- Committing a transaction
- Triggering a rollback on a transaction
:warn
- Forced transaction resets
Apart from the already mentioned ones, the following class specific options are available:
Determines whether petra
should automatically create proxies for instances of the configured class when they are accessed from within an existing object proxy.
Petra.configure do
configure_class SimpleUser do
proxy_instances true
end
# Do not create a proxy for strings. Otherwise, calling `SimpleUser#first_name` would result in a string object proxy
configure_class String do
proxy_instances false
end
end
petra
contains a very basic ObjectProxy
implementation which works fine with most ruby objects, but has to be configured.
For more advanced classes, it is advised to create a specialized proxy (see petra-rails
).
By default, petra
will use the specialized version if available, but can be forced to use the basic object proxy instead:
Petra.configure do
configure_class ActiveRecord::Base do
use_specialized_proxy false
end
end
petra
does not only support proxies for certain classes, but also for mixins. This allows a developer to define a proxy which is automatically used for every class which contains a certain module.
By default, petra
contains an Enumerable
proxy which automatically wraps its entries in object proxies.
The automatic inclusion of these module proxies can be disabled:
Petra.configure do
configure_class Array do
mixin_module_proxies false
end
end
Specifies the method to retrieve an identifier for instances of the configured class.
By default, object_id
is used, which of course is very limited.
Petra.configure do
configure_class ActiveRecord::Base do
id_method :id
# or
id_method do |obj|
obj.id
end
end
end
Basically the counterpart of id_method
. Specifies the class method which can be used to retrieve an instance of the configured class when providing the corresponding identifier.
It defaults to ObjectSpace._id2ref
which returns an object by its object_id
.
Petra.configure do
configure_class ActiveRecord::Base do
lookup_method :find
end
end
Specifies the method to initialize a new instance of the configured class (or one of its descendants).
It is used to automatically re-initialize objects used (and persisted) in a previous section and works the same way as lookup_method.
Petra.configure do
configure_class Array do
init_method :new
end
end
petra
can be easily extended to a certain extent as seen in stex/petra-rails.
As mentioned above, some classes are too complicated to be configured using the basic ObjectProxy
.
Let's define a basic example for such a class:
class SimpleRecord
def self.create(attributes = {})
new(attributes).save
end
def save
# some persistence logic
end
end
In this example, #create
is a method we cannot configure easily as it doesn't match any of the available method types in ObjectProxy
. Instead. it is a combination of attribute writers and persistence methods.
To be taken into account as a custom object proxy, a class has to comply to the following rules:
- It has to be defined inside
Petra::Proxies
- It has to inherit from
Petra::Proxies::ObjectProxy
- It has to define the class names it may be applied to in a constant named
CLASS_NAMES
Let's define the corresponding proxy for SimpleRecord
:
module Petra
module Proxies
class SimpleRecordProxy < ObjectProxy
CLASS_NAMES = %w[SimpleRecord].freeze
def create(attributes = {})
# This method may only be called on class, not on instance level
class_method!
# Use ObjectProxy's basic `new` method without any arguments
new.tap do |obj|
# Tell our transaction that we initialized a new object.
# This wasn't done in the previous examples as we were working on the
# `ObjectSpace` with objects defined outside the transaction.
transaction.log_object_initialization(o, method: 'new')
# Apply the attribute writes inside the transaction
attributes.each do |k, v|
__set_attribute(k, v)
end
# #create automatically persists a record, we therefore have to
# tell our transaction to log this action.
transaction.log_object_persistence(o, method: 'save')
end
end
def save
transaction.log_object_persistence(self, method: 'save')
end
end
end
end
See petra-rails's ActiveRecordProxy for a full example.
As mentioned above, module proxies can be used to define proxy functionality for all classes which include a certain module.
Internally, these modules are included into the singleton class of our object proxies, meaning that one instance of a proxy could include a certain module, the other doesn't.
A module proxy has to comply to the following rules:
- It has to be defined in
Petra::Proxies
- It has to include
Petra::Proxies::ModuleProxy
- It has to define a constant named
MODULE_NAMES
which contains the modules it is applicable for.
Let's take a look at petra
's EnumerableProxy
:
module Petra
module Proxies
module EnumerableProxy
include ModuleProxy
MODULE_NAMES = %w[Enumerable].freeze
# Specifying an `INCLUDES` constant leads to instances of the resulting proxy
# automatically including the given modules - in this case, every proxy which handles
# an Enumerable will automatically be an Enumerable as well
INCLUDES = [Enumerable].freeze
# ModuleProxies may specify an `InstanceMethods` and a `ClassMethods` sub-module.
# Their methods will be included/extended accordingly.
module InstanceMethods
#
# We have to define our own #each method for the singleton class' Enumerable
# It basically just wraps the original enum's entries in proxies and executes
# the "normal" #each
#
def each(&block)
Petra::Proxies::EnumerableProxy.proxy_entries(proxied_object).each(&block)
end
end
#
# Ensures the the objects yielded to blocks are actually petra proxies.
# This is necessary as the internal call to +each+ would be forwarded to the
# actual Enumerable object and result in unproxied objects.
#
# This method will only proxy objects which allow this through the class config
# as the enum's entries are seen as inherited objects.
# `[]` is used as method causing the proxy creation as it's closest to what's actually happening.
#
# @return [Array<Petra::Proxies::ObjectProxy>]
#
def self.proxy_entries(enum, surrogate_method: '[]')
enum.entries.map { |o| o.petra(inherited: true, configuration_args: [surrogate_method]) }
end
end
end
end
Please take a look at lib/petra/proxies/abstract_proxy.rb
for more information regarding how proxies are chosen and built.
For its transaction handling, petra
needs access to a storage with atomic write operations to store its transaction logs as well as being able to lock certain resources (during commit phase, no other transaction may have access to certain resources).
Petra::PersistenceAdapters::Adapter
provides an interface for classes which provide this functionality. FileAdapter
is the reference implementation which uses the file system and UNIX file locks.
persist!
Saves all available transaction log entries to the storage.
Log entries are added using #enqueue(entry)
and available as queue
inside your adapter instance.
- A transaction lock has to be applied
- Entries have to be marked as persisted afterwards using
entry.mark_as_persisted!
transaction_identifiers
Should return the identifiers of all transactions which were started, but not yet committed.
savepoints(transaction)
Should return all savepoints (section identifiers) for the given transaction,
log_entries(section)
Should return all log entries which were persisted for the given section in the past.
reset_transaction(transaction)
Removes all information currently stored regarding the given transaction
with_global_lock(suspend:, &block)
Acquires a global lock (only one thread may hold it at the same time), runs the given block and releases the global lock again.
If suspend
is set to true
, the execution will wait for the lock to be available, otherwise, a Petra::LockError
is thrown if the lock is not available.
You have to make sure that the lock is freed again if an error occurs within the given block or your own implementation.
with_transaction_lock(transaction, suspend:)
Acquires a lock on the given transaction.
with_object_lock(object, suspend:)
Acquires a lock on the given Object (Proxy).
Make sure that your implementation allows one thread locking the resource multiple times without stalling.
with_object_lock(obj1) do
with_object_lock(obj1) do # Should work as we already hold the lock
...
end
end
Similar to Rails' mailer adapters, new adapter can be registered under a given name and be used in petra
's configuration afterwards:
Petra::PersistenceAdapters::Adapter.register_adapter(:redis, RedisAdapter)
Petra.configure do
persistence_adapter :redis
end