petra

Petra is a proof-of-concept for persisted transactions in Ruby with (hopefully) full ACI(D) properties.

Please note that this was created during my master's thesis in 2016 and hasn't been extended a lot since then except for a few coding style fixes. I would write a lot of stuff differently today, but the main concept is still interesting enough.

It allows starting a transaction without committing it and resuming it at a later time, even in another process - given the used objects provide identifiers other than object_id.

It should work with every Ruby object and can be extended to work with web frameworks like Ruby-on-Rails as well (a POC of RoR integration can be found at stex/petra-rails).

This README only covers parts of what petra has to offer. Feel free to dive into the code, everything should be commented accordingly.

Let's take a look at how petra is used:

class SimpleUser
  attr_accessor :first_name, :last_name
  
  def name
    "#{first_name} #{last_name}"
  end
  
  # ... configuration, see below
end

user = SimpleUser.petra.new('John', 'Doe')

# Start a new transaction and start changing attributes
Petra.transaction(identifier: 'tr1') do
  user.first_name = 'Foo'
end

# No changes outside the transaction yet...
puts user.name #=> 'John Doe'

# Continue the same transaction
Petra.transaction(identifier: 'tr1') do
  puts user.name #=> 'Foo Doe'
  user.last_name = 'Bar'
end

# Another transaction changes a value already changed in 'tr1'
Petra.transaction do
  user.first_name = 'Moo'
  Petra.commit!
end

puts user.name #=> 'Moo Doe'

# Try to commit our first transaction
Petra.transaction(identifier: 'tr1') do
  puts user.name
  Petra.commit!
rescue Petra::WriteClashError => e
  # => "The attribute `first_name` has been changed externally and in the transaction. (Petra::WriteClashError)"
  # Let's use our value and go on with committing the transaction
  e.use_ours!
  e.continue!
end

# The actual object is updated with the values from tr1
puts user.name #=> 'Foo Bar'

We just used a simple Ruby object inside a transaction which was even split into multiple sections!

(The full example can be found at examples/showcase.rb)

TOC

• Installation
• Basic Usage
  • Starting/Resuming a transaction
  • Transactional Objects and their Configuration
  • Commit / Rollback / Reset / Retry
• Reacting to external changes
  • An attribute we previously read was changed externally
  • An attribute we changed in our transaction was also changed externally
  • continue!?
• Full Configuration Options
  • Global Options
  • Class Specific Options
• Extending petra
  • Class Proxies
  • Module Proxies
  • Persistence Adapters

Installation

Simply add the following line to your gemfile:

gem 'petra_core', require: 'petra'

Unfortunately, the gem name petra is already taken and petra-core would express that this gem is extending it, so I went for an underscore for now. It's hard finding nice-sounding gem names which are not yet taken nowadays :/

Basic Usage

Starting/Resuming a transaction

Whenver you call Petra.transaction, a transaction section is started. If you pass in an identifier and a matching transaction already exists, it will be resumed instead.

# Starting a new transaction with an auto-generated identifier
tr_id = Petra.transaction {}

# Resuming the transaction
Petra.transaction(identifier: tr_id) {}

Transactional Objects and their Configuration

Although petra is seemingly able to use every Ruby object inside a transaction, it does not patch these objects in any way by e.g. overriding their getters and setters. Instead, a transparent proxy is used:

# Normal instance of SimpleUser
user = SimpleUser.new

# ObjectProxy, can now be used inside and outside of transactions
user = SimpleUser.petra.new # or: user = SimpleUser.new.petra

In its current version, petra has to be told about the meaning of the different methods of a class to be used inside a transaction.
This decision was made as there are no strict conventions regarding method names in Ruby (e.g. getX/setX in Java).

petra knows about 5 different kinds of methods:

1. Attribute Readers which retrieve a current attribute value
2. Attribute Writers which set a new attribute value
3. Dynamic Attribute Readers which a composite methods like name (not an actual attribute, but use attributes interally)
4. Persistence Methods which save changes made to the object (think of ActiveRecord::Base#save)
5. Destruction Methods which remove the object

Let's create a configuration for SimpleUser:

Petra.configure do
  configure_class SimpleUser do
    # Tell petra about our available attribute readers
    attribute_reader? do |method_name|
      %w[first_name last_name].include?(method_name.to_s)
    end
    
    # Do the same for attribute writers
    attribute_writer? do |method_name|
      %w[first_name= last_name=].include?(method_name.to_s)
      # also possible here: `method_name.last == '='`
    end    
    
    # Define which methods are used to persist instances of SimpleUser
    persistence_method? do |method_name|
      %w[first_name= last_name=].include?(method_name.to_s)
    end    

    # `name` uses attributes internally
    dynamic_attribute_reader? do |method_name|
      %[name].include?(method_name.to_s)
    end
  end
end

As you may have noticed, we used our attribute_writers twice in this configuration: Once as actual attribute writers and once as persistence method. This was done to keep the example above as small as possible.

The same could have been achieved by setting up a no-op method and configuring it accordingly:

# SimpleUser
def save; end

# Configuration
persistence_method { |method_name| %w[save].include?(method_name.to_s) } 

# Usage
Petra.transaction do
  user.first_name = 'Foo'
  user.save
end

In this case, not calling save inside the transaction would have lead to the loss of everything we did inside the transaction section.

Commit / Rollback / Reset / Retry

Commit

Transactions can be committed by calling Petra.commit! inside a Petra.transaction block.
It will leave the transaction block afterwards and not execute anything left in it:

Petra.transaction do
  Petra.commit!
  puts 'I will never be shown!'
end

Rollback

A rollback can be triggered by either raising Petra::Rollback or simply any other uncaught StandardError. The difference is that Petra::Rollback will be swallowed by the transaction processing (like ActiveRecord::Rollback does), while any other error will be re-raised.

Triggering a rollback will undo all changes made in the current section of the transaction. All previous sections are not affected.

Petra.transaction(identifier: 'tr1') do
  user.first_name = 'Foo'
end

Petra.transaction(identifier: 'tr1') do
  user.last_name = 'Bar'
  fail Petra::Rollback
end

In this example, only the change to user#last_name is lost.

Reset

A reset can be triggered by raising Petra::Reset. It works like a rollback, but will clear the whole transaction.

Petra.transaction(identifier: 'tr1') do
  user.first_name = 'Foo'
end

Petra.transaction(identifier: 'tr1') do
  user.last_name = 'Bar'
  fail Petra::Reset
end

Here, all changes to user are lost.

Retry

A retry means that the current transaction block should be retried again after a rollback.

Petra.transaction(identifier: 'tr1') do
  user.last_name = 'Bar'
  fail Petra::Retry if some_condition
end

Reacting to external changes

As the transaction is working in isolation on its own data set, it might happen that the original objects outside the transaction are changed in the meantime, e.g. by another transaction's commit:

Petra.transaction(identifier: 'tr1') do
  user.first_name = 'Foo'
end

Petra.transaction(identifier: 'tr2') do
  user.first_name = 'Moo'
  Petra.commit!
end

Petra.transaction(identifier: 'tr1') do
  # we don't know about the external change here and would
  # possibly override it
end

petra reacts to these external changes and raises a corresponding exception. This exception allows the developer to solve the conflicts based on his current context.

The exception is thrown either when the attribute is used again or during the commit phase. Not handling any of these exception yourself will result in a transaction reset.

Each error described below shares a few common methods to control the further transaction flow:

Petra.transaction(identifier: 'tr1') do
  begin
    ...
  rescue Petra::ValueComparisionError => e # Superclass of ReadIntegrityError and WriteClashError
    e.object          #=> the object which was changed externally
    e.attribute       #=> the name of the changed attribute
    e.external_value  #=> the new external value
  
    e.retry!    # Runs the current transaction block again
    e.rollback! # Dismisses all changes in the current section, continues after transaction block
    e.reset!    # Resets the whole transaction, continues after transaction block
    e.continue! # Continues with executing the current transaction block
  end
end

Please note that in most cases calling rollback!, retry! or continue! without any other exception specific method will result in the same error again the next time.

An attribute we previously read was changed externally

A ReadIntegrityError is thrown if one transaction reads an attribute value which is then changed externally:

Petra.transaction(identifier: 'tr1') do
  user.last_name = 'the first' if user.first_name = 'Karl'
end

user.first_name = 'Olaf'

Petra.transaction(identifier: 'tr1') do
  user.first_name 
  #=> Petra::ReadIntegrityError: The attribute `first_name` has been changed externally.
end

When triggering a ReadIntegrityError, you can choose to acknowledge/ignore the external change. Doing so will suppress further errors as long as the external value does not change again.

begin
...
rescue Petra::ReadIntegrityError => e
  e.last_read_value #=> the value we got when last reading the attribute

  e.ignore!(update_value: true)  # we acknowledge the external change and use the new value in our transaction from now on
  e.ignore!(update_value: false) # we keep our old value and simply ignore the external change.
  e.retry!
end

An attribute we changed in our transaction was also changed externally

A WriteClashError is thrown whenever an attribute we changed inside one of our transaction sections was also changed externally:

Petra.transaction(identifier: 'tr1') do
  user.first_name = 'Foo'
end

user.first_name = 'Moo'

Petra.transaction(identifier: 'tr1') do
  user.first_name
  #=> Petra:WriteClashError: The attribute `first_name` has been changed externally and in the transaction.
end

As both sides changed the attribute value, we have to decided which one to use further in most cases (or completely reset the transaction):

begin
...
rescue Petra::WriteClashError => e
  e.our_value   #=> the value we set the attribute to
  e.their_value #=> the new external value

  e.use_theirs! # undo every change we made to the attribute in this transaction
  e.use_ours!   # Ignore the external change, use our value
  e.retry!
end

continue!?

As mentioned above, petra allows the developer to jump back into the transaction after an error was resolved.
This is done by using Ruby's Continuation which basically saves a copy of the stack at the time the exception happened. This copy can then be restored if the developer decides to continue the execution.

I'd personally keep everything regarding continuations far away from production code, but they are a very interesting concept (which will most likely be removed with Ruby 3.0 :/ ). examples/continuation_error.rb shows one of the drawbacks which could lead to a long time of debugging.

begin
  simple_user.first_name = 'Foo'
  simple_user.save
rescue Petra::WriteClashError => e
  e.use_ours!
  # Jumps back to `simple_user.save` without a retry
  e.continue!
end

Full Configuration Options

Global Options

persistence_adapter

Petra.configure do
  persistence_adapter :file
  persistence_adapter.storage_directory = '/tmp/petra'
end

Specifies the persistence adapter and its possible options.
Petra only includes a file system based adapter by default.

instantly_fail_on_read_integrity_errors

Petra.configure do
  instantly_fail_on_read_integrity_errors false
end	

petra can be set to optimistic transaction handling. This means, that a transaction is only checked for possible external changes during the commit phase.

By default, a corresponding error is thrown directly when the attribute is accessed again within the transaction.

log_level

Petra.configure do
  log_level :debug | :info | :warn | :error
end

Specifies the log level petry should use.

• :debug
  • Information about all methods called on an object proxy and their results
  • Attribute reads and changes
  • Acquired and released locks
  • The creation of transaction log entries
• :info
  • Starting and persisting a transaction
  • Committing a transaction
  • Triggering a rollback on a transaction
• :warn
  • Forced transaction resets

Class Specific Options

Apart from the already mentioned ones, the following class specific options are available:

proxy_instances

Determines whether petra should automatically create proxies for instances of the configured class when they are accessed from within an existing object proxy.

Petra.configure do
  configure_class SimpleUser do
    proxy_instances true
  end
  
  # Do not create a proxy for strings. Otherwise, calling `SimpleUser#first_name` would result in a string object proxy
  configure_class String do
    proxy_instances false
  end
end

use_specialized_proxy

petra contains a very basic ObjectProxy implementation which works fine with most ruby objects, but has to be configured.
For more advanced classes, it is advised to create a specialized proxy (see petra-rails).

By default, petra will use the specialized version if available, but can be forced to use the basic object proxy instead:

Petra.configure do
  configure_class ActiveRecord::Base do
    use_specialized_proxy false
  end
end

mixin_module_proxies

petra does not only support proxies for certain classes, but also for mixins. This allows a developer to define a proxy which is automatically used for every class which contains a certain module.

By default, petra contains an Enumerable proxy which automatically wraps its entries in object proxies.

The automatic inclusion of these module proxies can be disabled:

Petra.configure do
  configure_class Array do
    mixin_module_proxies false
  end
end

id_method

Specifies the method to retrieve an identifier for instances of the configured class.

By default, object_id is used, which of course is very limited.

Petra.configure do
  configure_class ActiveRecord::Base do
    id_method :id
    # or
    id_method do |obj|
      obj.id
    end
  end
end

lookup_method

Basically the counterpart of id_method. Specifies the class method which can be used to retrieve an instance of the configured class when providing the corresponding identifier.

It defaults to ObjectSpace._id2ref which returns an object by its object_id.

Petra.configure do
  configure_class ActiveRecord::Base do
    lookup_method :find
  end
end

init_method

Specifies the method to initialize a new instance of the configured class (or one of its descendants).
It is used to automatically re-initialize objects used (and persisted) in a previous section and works the same way as lookup_method.

Petra.configure do
  configure_class Array do
    init_method :new
  end
end

Extending petra

petra can be easily extended to a certain extent as seen in stex/petra-rails.

Class Proxies

As mentioned above, some classes are too complicated to be configured using the basic ObjectProxy.

Let's define a basic example for such a class:

class SimpleRecord
  def self.create(attributes = {})
    new(attributes).save
  end
  
  def save
    # some persistence logic
  end
end

In this example, #create is a method we cannot configure easily as it doesn't match any of the available method types in ObjectProxy. Instead. it is a combination of attribute writers and persistence methods.

To be taken into account as a custom object proxy, a class has to comply to the following rules:

1. It has to be defined inside Petra::Proxies
2. It has to inherit from Petra::Proxies::ObjectProxy
3. It has to define the class names it may be applied to in a constant named CLASS_NAMES

Let's define the corresponding proxy for SimpleRecord:

module Petra
  module Proxies
    class SimpleRecordProxy < ObjectProxy
      CLASS_NAMES = %w[SimpleRecord].freeze

      def create(attributes = {})
        # This method may only be called on class, not on instance level
        class_method!

        # Use ObjectProxy's basic `new` method without any arguments
        new.tap do |obj|
          # Tell our transaction that we initialized a new object.
          # This wasn't done in the previous examples as we were working on the
          # `ObjectSpace` with objects defined outside the transaction.
          transaction.log_object_initialization(o, method: 'new')

          # Apply the attribute writes inside the transaction
          attributes.each do |k, v|
            __set_attribute(k, v)
          end

          # #create automatically persists a record, we therefore have to
          # tell our transaction to log this action.
          transaction.log_object_persistence(o, method: 'save')
        end
      end

      def save
        transaction.log_object_persistence(self, method: 'save')
      end
    end
  end
end

See petra-rails's ActiveRecordProxy for a full example.

Module Proxies

As mentioned above, module proxies can be used to define proxy functionality for all classes which include a certain module.
Internally, these modules are included into the singleton class of our object proxies, meaning that one instance of a proxy could include a certain module, the other doesn't.

A module proxy has to comply to the following rules:

1. It has to be defined in Petra::Proxies
2. It has to include Petra::Proxies::ModuleProxy
3. It has to define a constant named MODULE_NAMES which contains the modules it is applicable for.

Let's take a look at petra's EnumerableProxy:

module Petra
  module Proxies
    module EnumerableProxy
      include ModuleProxy
      MODULE_NAMES = %w[Enumerable].freeze

      # Specifying an `INCLUDES` constant leads to instances of the resulting proxy
      # automatically including the given modules - in this case, every proxy which handles
      # an Enumerable will automatically be an Enumerable as well
      INCLUDES = [Enumerable].freeze

      # ModuleProxies may specify an `InstanceMethods` and a `ClassMethods` sub-module.
      # Their methods will be included/extended accordingly.
      module InstanceMethods
        #
        # We have to define our own #each method for the singleton class' Enumerable
        # It basically just wraps the original enum's entries in proxies and executes
        # the "normal" #each
        #
        def each(&block)
          Petra::Proxies::EnumerableProxy.proxy_entries(proxied_object).each(&block)
        end
      end

      #
      # Ensures the the objects yielded to blocks are actually petra proxies.
      # This is necessary as the internal call to +each+ would be forwarded to the
      # actual Enumerable object and result in unproxied objects.
      #
      # This method will only proxy objects which allow this through the class config
      # as the enum's entries are seen as inherited objects.
      # `[]` is used as method causing the proxy creation as it's closest to what's actually happening.
      #
      # @return [Array<Petra::Proxies::ObjectProxy>]
      #
      def self.proxy_entries(enum, surrogate_method: '[]')
        enum.entries.map { |o| o.petra(inherited: true, configuration_args: [surrogate_method]) }
      end
    end
  end
end

Please take a look at lib/petra/proxies/abstract_proxy.rb for more information regarding how proxies are chosen and built.

Persistence Adapters

For its transaction handling, petra needs access to a storage with atomic write operations to store its transaction logs as well as being able to lock certain resources (during commit phase, no other transaction may have access to certain resources).

Petra::PersistenceAdapters::Adapter provides an interface for classes which provide this functionality. FileAdapter is the reference implementation which uses the file system and UNIX file locks.

Required Methods

persist!

Saves all available transaction log entries to the storage. Log entries are added using #enqueue(entry) and available as queue inside your adapter instance.

• A transaction lock has to be applied
• Entries have to be marked as persisted afterwards using entry.mark_as_persisted!

transaction_identifiers

Should return the identifiers of all transactions which were started, but not yet committed.

savepoints(transaction)

Should return all savepoints (section identifiers) for the given transaction,

log_entries(section)

Should return all log entries which were persisted for the given section in the past.

reset_transaction(transaction)

Removes all information currently stored regarding the given transaction

with_global_lock(suspend:, &block)

Acquires a global lock (only one thread may hold it at the same time), runs the given block and releases the global lock again.

If suspend is set to true, the execution will wait for the lock to be available, otherwise, a Petra::LockError is thrown if the lock is not available.

You have to make sure that the lock is freed again if an error occurs within the given block or your own implementation.

with_transaction_lock(transaction, suspend:)

Acquires a lock on the given transaction.

with_object_lock(object, suspend:)

Acquires a lock on the given Object (Proxy).

Make sure that your implementation allows one thread locking the resource multiple times without stalling.

with_object_lock(obj1) do
  with_object_lock(obj1) do # Should work as we already hold the lock
   ...
  end
end 

Registering a new adapter

Similar to Rails' mailer adapters, new adapter can be registered under a given name and be used in petra's configuration afterwards:

Petra::PersistenceAdapters::Adapter.register_adapter(:redis, RedisAdapter)

Petra.configure do
  persistence_adapter :redis
end