CHANGES.txt

================================ Next release ================================

========================== Release 0.7.0 2015-08-07 ==========================

Upgraded to Twisted 15.3, updated Service subclasses to reflect the new-style
Service classes in this version.

========================== Release 0.6.0 2015-05-29 ==========================

This pertains to piped.contrib.database only:

Backward-incompatible changes:
    piped.contrib.database's SerialPostgresListenerService is removed.
    The remaining PostgresListenerService is also simpler, not caring about
    txid_min thresholds.

Bug fixes:
    - Reconnect Postgres listeners more reliably.
    - Don't cause multiple run_as_leader run-loops to be spawned on certain
      reconnects.

========================== Release 0.6.0 2015-02-20 ==========================

Features:
    ``piped`` now supports passing ``--pidfile=`` (an empty string) which
    in turn causes ``twistd`` to not write a pid file.

Backward-incompatible changes:
    piped.contrib.zookeeper version 0.5.10 no longer accepts 'default_acls'
      and now requires acls to be pre-encoded using the 'default_encoded_acls'
      argument instead.

Contrib:
    piped.contrib.zookeeper: version 0.5.8 fixes an issue where the zookeeper
      default open acl were used if only encoded acls were provided.
    piped.contrib.zookeeper: version 0.5.9 fixes an issue where the zookeeper
      client was provided before authing had completed

========================== Release 0.5.9 2015-01-27 ==========================

Workaround:
    - Will temporarily add setContent to twisted.python.zippath.ZipPath while
      looking for plugins to avoid an excessive amount of logging.

Bug fixes:
    - Avoid disabling existing loggers if they are created before the logging
      system was initialized, which might happen if they're imported first.

========================== Release 0.5.8 2014-04-18 ==========================

Bug fixes:
    - Memory leak when making loggers per context-specific logger system, like
      e.g. HTTPConnections or SSHServerTransports.

    - InstanceDependency has been given a ``__hash__`` and ``__eq__`` method
      that delegates to the underlying instance.

    - DependencyMap stays an InstanceDependency but uses the built-in
      ``object`` definitions for its ``__hash__`` and ``__eq__`` methods.

========================== Release 0.5.7 2014-03-24 ==========================

Features:
    - The zookeeper-contrib now accepts a parameter, ``default_encoded_acls``,
      which is expected to be a list of pre-encoded ACLs. This is the
      recommended way of passing default ACLs to clients provided by this
      provider since it doesn't need to know the exact passwords for these
      ACLs, just their base64-encoded hashes.

========================== Release 0.5.6 2013-04-09 ==========================

Bug fixes:
    - A regression in 0.5.5 broke customization of logging.

========================== Release 0.5.5 2013-04-09 ==========================

Bug fixes:
    - Specifying a custom reactor with the "-r"-argument to piped should now
      work again.

Features:
    - The constructor of ``piped.event.Event`` now takes a single keyword
      argument: ``async``, which, if ``True`` makes ``.__call__`` return
      a deferred which callbacks when all the listeners have finished
      processing the event.

========================== Release 0.5.3 2013-01-06 ==========================

Features:
    - New StatsD-contrib that handles one or more statsd-connections, as well
      as a reporter-service for various process metrics.

    - ZooKeeper-contrib now authenticates and creates nodes with default ACLs.

    - ZooKeeper-client learned create_recursive.


========================== Release 0.5.2 2012-12-01 ==========================

Features:
    - PostgresListener now knows about advisory locks.

    - The database-contrib now has a service-module with two
      PostgresListener-base-classes, for making services that should react to
      Postgres-notifications.

    - ``piped.util`` gained a contextual ``DeferredLock`` and a
      ``BackoffWaiter``.

Bug fixes:
    - ``piped.contrib.zookeeper`` attempted to log a line with formatting errors
      after connecting to ZooKeeper.

========================== Release 0.5.1 2012-11-16 ==========================

Bug fixes:
    - ``p.util.Cancellable.disown_cancellable_parent_callback`` now correctly
      passes the return value through.

    - The ZooKeeper provider should now parse multiple servers with chroots
      correctly.

========================== Release 0.5.0 2012-09-23 ==========================

Features:
    - Plugin-loaded services: ``IPipedService``-plugins, with a helper
      ``PipedService``-base class in ``piped.service``. If the service uses
      dependencies from other providers, consider using
      ``DependencyPipedService``.

Internal changes:
    - ``piped.util.wait(..)`` is now cancellable.
    - ``DependencyManager.add_dependency(foo)`` now returns ``foo`` as a
     dependency after binding it to the dependency graph instead of returning
     ``None``.

    - The database contrib has been reworked. By just providing
      SQLAlchemy engines instead of MetaData, it's a lot simpler. The
      API is still unstable, though.

    - A PostgresListener-service makes working with asynchronous
      notifications from a Postgres database a lot simpler. It also
      has utilities for watching transaction snapshot threshold, which
      is useful when implementing certain syncing strategies.

Backward-incompatible changes:

    - The database-contrib now provides and manages SQLAlchemy
      engines, instead of ``MetaData``.

Bug fixes:
    - The ZooKeeper provider could sometimes fail connecting while still
     providing the client.
    - ``piped.util.wait_for_first`` will now errback with the actual failure,
     and not a ``FirstError``.
    - ``piped.graph`` is now compatible with ``networkx`` 1.7


========================== Release 0.4.7 2012-08-09 ==========================

Features:
    - The ZooKeeper contrib now uses a configurable ``connect_timeout`` and a
      configurable ``reconnect_timeout``, which is used when connecting and
      reconnecting, respectively.

    - Added a setting ``force_blocking_stdout`` to the REPL provider, which
      makes ``sys.__stdout__`` blocking after creating the StandardIO
      interface. This is done by default because a lot of our logging seem
      to enjoy a blocking standard output a lot more than handling EAGAIN.

Internal changes:
    - The ZooKeeper contrib will now retry timed out connections from python,
      trying different combinations of servers in the specified server list
      until one of the combinations starts working.

    - Added `piped.util.deferred_with_timeout`` which is a function that
      wraps a deferred with an optional timeout.

Minor bug fixes:
    - A traceback from the bootstrap script to the REPL should now show
      the actual line contents from the source script.


========================== Release 0.4.6 2012-08-08 ==========================

Bug fixes:
    - Default logging level should be INFO, not DEBUG.

========================== Release 0.4.5 2012-08-08 ==========================

Features:
    - REPL: Piped now comes with a built-in REPL (a Read-Eval-Print loop,
      similar to the Python shell). To start the REPL, use the "--repl"
      command line switch. For more information about the REPL and how
      to configure it, see the reference documentation on the REPL provider,
      which contains example configuration in YAML.

Bug fixes:
    - The bundled logging configuration should no longer emit duplicate log
      messages.

========================== Release 0.4.4 2012-06-12 ==========================

Bug fixes:
    - No longer requires graypy by default via the logging configuration.
    - Buildout now refers to the correct contrib package names.

========================== Release 0.4.3 2012-06-12 ==========================

Internal changes:
    - The ZooKeeper contrib is more determined about closing previous
      connections to avoid leaking unused clients and clears its cache
      when connections are re-established.

Backward-incompatible changes:
    - The ZooKeeper contrib now raises zookeeper.ClosingException exceptions
      for attempted method calls to a stopped client instead of the previous
      AttributeError

========================== Release 0.4.2 2012-02-29 ==========================

Internal changes:
    - The bundled logging.yaml configuration file now defaults to "INFO"
      instead of "DEBUG"

    - The Cyclone contrib now includes XSRF-tokens in the debugging template.

Backward-incompatible changes:
    - The ZooKeeper contrib now proxies a client instead of subclassing it
      in order to more easily facilitate reconnects.

========================== Release 0.4.0 2012-02-26 ==========================

Backward-incompatible changes:
    - piped.log is deprecated and the standard Python logging module should
      be used instead.

    - ``piped -l`` now no longer is the name of a file to log to, but the name
      of a logging configuration file. See the ``logging.yaml`` file included
      in the piped distribution for example usage.


========================== Release 0.3.2 2012-02-18 ==========================

Internal changes:
    - piped.dependencies.DependencyMap now has a ``.get_dependency(key)`` method
      that returns the dependency object for the given key in the dependency
      map.


========================== Release 0.3.1 2012-02-15 ==========================

Features:
    - ZooKeeper: A ZooKeeper-contrib is now available, for interacting
      with ZooKeeper ensembles. In addition to a ZooKeeper-client
      provider, a few processors were added as well.

    - AMQP: Support for providing AMQP-connections, consuming messages
      and processing them in pipelines, as well as a simple helper
      for creating and providing AMQP-based RPC-clients.

    - Cyclone: Support for Cyclone applications (http://cyclone.io), which
      is an implementation of Tornado (http://tornadoweb.org) on top of
      Twisted. This enables building full-fledged web-apps that use the
      dependency/resource management/pipelines in piped.

    - Built-in and contrib providers now use "processor: ..." instead
        of "pipeline: ...", and are able to depend on arbitrary dependencies.
        The dependencies are invoked with the baton as the first and
        only argument.

        This makes it easier for one provider to call a function on another
        provider.

    - Different twisted reactors can now be selected by using the
      "-r/--reactor" arguments to the "piped" script. See "--help-reactors"
      for a description of the available reactors.

    - The web-provider now supports "listen" as a an optional parameter
      instead of "port". The new "listen" parameter is a "strport", which
      enables listening to single interface / configuring SSL sites etc.
      For more information about strports, see
      http://twistedmatrix.com/documents/current/api/twisted.application.strports.html

    - It is now possible to check whether a "piped.dependencies.DependencyMap" contains
      a dependency (which may or may not currently be available) by using the "in"
      operator.

    - "piped.event.Event.wait_until_fired()" now takes an optional "timeout" argument.
      If the event has not fired before the timeout is reached, a
      "piped.exceptions.TimeoutError" is raised. The default timeout of ``None`` means
      no timeout.

    - "InstanceDependency", "ResourceDependency" and "DependencyMap" in
      "piped.dependencies" now has support for a timeout keyword argument to
      ".wait_for_resource()". If the resource is not available before the timeout
      is reached, a "piped.exceptions.TimeoutError" is raised.

    - encode-json now passes the `indent`-option to `json.dumps`.

    - Configuration includes now support including files at a sub-path in the
      current configuration. See the configuration documentation for more details.


New processors:
    - get-zookeeper-children: Get a list of children for a ZooKeeper node.
    - get-zookeeper-data: Get the data of a ZooKeeper node.
    - zookeeper-node-exists: Check whether a given ZooKeeper node exists.
    - set-zookeeper-data: Set the contents of a ZooKeeper node.
    - create-zookeeper-node: Create a ZooKeeper node.
    - web-client-get-page: Getting a web page.


Internal changes:
    - piped.processing now uses stores the processor name under the
        '__processor__' key instead of 'processor', enabling processors to
        take 'processor' as a keyword argument.

    - piped and contrib plugins no longer use namespace packages, but uses the
        same scheme as twisted for loading external plugins. This means that
        plugins (both processors and providers) are automatically found if they
        are available in a module or package under the folder "piped/plugins"
        anywhere on sys.path.

    - DummyRequest moved back from web_provider to test_web_provider as it uses
        code from the twisted test suite, which shouldn't be used in production.


Backward-incompatible changes:
    - The syntax for creating processors in the YAML configuration has changed.
        Using "processor: processor_name" is no longer supported. Use

            - processor_name

            OR

            - processor_name:
                foo: bar

            OR

            - __processor__: processor_name
              foo: bar

         instead.

    - The processors proxy-forward and chain-web-requests were removed because
        they used code from the twisted test suite, which shouldn't be used in
        production. They might come back at a later time, but no time-frame can
        be given at this moment.

    - "piped.dependencies.DependencyMap.__in__" has been removed. Use
      ".is_available" instead.

    - trap-failure now sets the Failure-object instead of just the exception-type.


Bug fixes:
    - #21: When tracing, if a processor asynchronously raised an exception,
      the trace token could be lost, causing the trace to be incomplete.

    - #18: Twisted 11.1 changed some default behaviour, causing debug-mode
       and tracing to break.

    - #12: pip may now be used to install/uninstall both piped and contrib
        packages.

    - #19: DummyRequest are now only used in the tests, in order to avoid using
        code from the twisted test suite in production.

    - dict_set_path did not deal with dict-lookups causing AttributeErrors.


========================== Release 0.2.0 2011-10-17 ==========================

Features:
    - Tracing: Processing in pipelines can now be traced. See the tracing
        tutorial.

    - web_provider: The web resources now support an additional setting in
        the routing configuration: "no_resource_pipeline", which acts like
        catch-all pipelines that is used to render requests for potentially
        nested child resources that does not have any resource (and would
        otherwise have resulted in a 404).

    - Support for acting like a Proxy. (see the proxy-forward processor).

    - Added a new YAML constructor: !path, which is used to refer to input
        that should be retrieved from the specified path inside the baton.

        It is intended to be used by processors to avoid having to have two
        separate keywords for input that either come from the configuration
        or from the current baton.

        See processors.base.Processor.get_input(...).

    - YAML-configurations can now be executed by piped like scripts using
        the shebang: #!/usr/bin/env piped -ns

    - chained_consumers and chained_error_consumers may now occur at any
        level of nesting within the pipeline configuration. Previously,
        chained_error_consumers were limited to the root level.

    - Made the ordering of consumers vs chained_consumers explicit:
        chained_consumers now always take precedence over consumers. See
        the pipelines topic page for more details.

Bug fixes:
    - Allow loops in the processor graphs to include the source processors.
    - fixed a bug that resulted in processors not having
        instance_depends_on/instance_provides.
    - web_provider: Fixed a bug that caused the automatic closing of http
        requests to stop working if a sink in the pipeline returned the
        request.

Changed processors:
    - decode-json: Added support for decoding from file-like objects.
    - for-each: Now iterates over values when processing a dictionary, returning
        a new dictionary with the original keys mapping to their respective processed
        values.
    - run-pipeline: Can now get the name of the pipeline to invoke from the
        baton.

New processors:
    - call-dependency: Calls a method on a dependency.
    - chain-web-requests: Chains two web requests together. See proxy-forward.
    - extract-web-request-arguments: Extract query string arguments from a
        twisted.web.server.Request-like object.
    - log: Logs a message with the configured log-level.
    - prefix-string: Prefixes the string at *input_path* with the
        *prefix*.
    - proxy-forward: Forwards requests to another server.
    - trap-failure: Traps failures of the specified types.

Internal changes:
    - The dependencies of an object can now be retrieved from the
        DependencyManager.
    - util.BatonJSONEncoder can be used to encode virtually any object
        to JSON by falling back to using the result of repr(obj) if it
        cannot otherwise be encoded.
    - __getitem__ has been added to evaluators and processor graphs allowing
        direct access to their processors.
    - ConcatenatedFile is no longer a subclass of StaticFile, but rather a
        twisted.web.resource.Resource subclass.
    - DummyRequest moved from test_web_provider to web_provider, as it might
        prove useful outside of