The format is based on Keep a Changelog. It uses CalVer as of May 2019.
24.11 - 2024-11-05
- Updated the gems.
- Added a pagination to the
Github
notices importer.
24.09 - 2024-09-12
- Created an importer of GitHub notices.
- Updated the gems.
- Allowed to search by the
date_submitted
(created_at
) field.
24.06a - 2024-06-07
- Made the submitter widget operational again.
24.06 - 2024-06-07
- Fixed translations related to the submitter widget.
- Fixed multiple issues with the submitter widget.
24.05 - 2024-05-31
- Allowed to make exect searches.
- Made logged-in users able to watch notices for new documents.
- Applied security updates.
24.03b - 2024-03-29
- Set the rack's
multipart_total_part_limit
setting to unlimited.
24.03a - 2024-03-27
- Applied a security update to the
rdoc
gem.
24.03 - 2024-03-15
- Stopped strictly validating urls, allowing any (redacted) string.
- Applied security updates.
24.02a - 2024-02-20
- Updated the
nokogiri
gem.
- Fixed a position of the magnifier icon in the advanced search form.
24.02 - 2024-02-15
- Allowed longer notices title.
- Stopped caching for super admins.
- Allowed entities to be linked with many users.
- Made sure an entity linked with submitter users is never redacted.
24.01a - 2024-01-12
- Upgraded
Rails
to7.2
.
24.01 - 2024-01-05
- Started using
Ruby 3.2
.
23.12b - 2023-12-19
- Enabled the
entities_country_codes
search field. - Set the default notice jurisdiction to the sender country code when not explicitly provided.
23.12a - 2023-12-15
- Stopped redacting phone numbers in defamation urls.
23.12 - 2023-12-13
- Installed and configured
Sidekiq
for background jobs. - Added a module for clearing front-end caches like
Varnish
.
23.11 - 2023-11-20
- Refactored redactors.
23.09a - 2023-09-11
- Started using Logstash for Elasticsearch indexing.
23.09 - 2023-09-04
- Updated the
rails
gem.
23.07 - 2023-07-28
- Updated the
rails
andwebrick
gems.
23.04b - 2023-04-26
- Started using a new
JSON parser
calledoj
. - Started a
Gemfile
cleanup.
23.04a - 2023-04-17
- Updated minor versions of
nokogiri
,rails_admin
,devise
andsimple_form
.
23.04 - 2023-04-06
- Updated the
rails
andrack
gem.
23.03a - 2023-03-10
- Updated the
rails
andrack
gem.
23.03 - 2023-03-10
- Allowed to use duplicated entities when creating a new notice.
23.02 - 2023-02-20
- Fixed an issue with sending out notifications about new court order reports.
23.01a - 2023-01-19
- Updated the
rails
gem.
23.01 - 2023-01-03
- Removed unused database tables.
- Added a new scope to the list
notices
view in theadmin
to be able to filter out notices with attachments only.
22.12c - 2022-12-15
- Set default values for
boolean
db fields.
22.12b - 2022-12-13
- Stopped running
.count
on associated models in theedit view
in theadmin
.
22.12a - 2022-12-08
- Made times in lists in the
admin
respect the custom timezone.
22.12 - 2022-12-08
- Upgraded the
rails_admin
gem to3.x
.
22.11b - 2022-11-30
- Allowed to destroy file uploads using notice nested attributes.
22.11a - 2022-11-28
- Limited metrics logging.
22.11 - 2022-11-17
- Stopped using a fork of the
nested_form
gem.
22.10b - 2022-10-25
- Made the
EmailRedactor
stop redacting urls.
22.10a - 2022-10-24
- Updated multiple gems.
22.10 - 2022-10-04
- Added a request id to log messages.
22.09h - 2022-09-30
- Made the application possible to log in the
Logstash
format.
22.09g - 2022-09-28
- Made url fields in the
admin
render astextarea
.
22.09f - 2022-09-27
- Made
textarea
fields in theadmin
wider generally.
22.09e - 2022-09-23
- Enlarge the
work description
fields in theadmin
. - Made
search facets
work properly in themedia mentions
view.
22.09d - 2022-09-21
- Moved the
site language
to thedatabase
.
22.09c - 2022-09-14
- Added
request
information to404s log messages
.
22.09b - 2022-09-13
- Set the proper type for
jsonb
fields to correctly render in theadmin
.
22.09a - 2022-09-05
- Fixed duplicates of
special domains
when showing the full notice version.
22.09 - 2022-09-05
- Introduced special domains.
22.08d - 2022-08-31
- Fixed double-redaction of senders/principals in defamations.
22.08c - 2022-08-30
- Started redacting sender and principal names in defamations.
22.08b - 2022-08-26
- Started catching more 404 errors.
22.08a - 2022-08-26
- Started properly catching requests with null bytes.
22.08 - 2022-08-02
- Created a system status page.
22.07i - 2022-07-28
- Set max width of system tooltips.
22.07h - 2022-07-28
- Forced a
submitter role entity
to always be a linked entity of a submitting user.
22.07g - 2022-07-27
- Added a
rake
task for merging similar entities.
22.07f - 2022-07-22
- Put the
name_original
field back to theentity edit form
in theadmin
.
22.07e - 2022-07-21
- Added a tooltip next to the
entity name
in thenotice view
showing additional entity information.
22.07d - 2022-07-19
- Disallow to download attachments of hidden notices.
22.07c - 2022-07-08
- Fixed removing
file uploads
when creatingnew notices
in theadmin
.
22.07b - 2022-07-08
- Made it possible to customize the
notice view
per notice.
22.07a - 2022-07-07
- Made the json editor in the admin show all work properties.
22.07 - 2022-07-07
- Made the admin render a json editor instance for new notices.
- Updated the
rails-html-sanitizer
gem.
22.06c - 2022-06-24
- Made the
JSON editor
in theadmin
work with sub-models of thenotice
model.
22.06b - 2022-06-24
- Stopped exposing full urls in the search view.
22.06a - 2022-06-07
- Moved taggings to new
jsonb
columns in thenotices
table.
22.06 - 2022-06-02
- Started moving existing taggings to new
jsonb
fields in thenotice
table.
22.05c - 2022-05-31
- Added a db constraint to avoid inserting wrong works json data.
22.05b - 2022-05-13
- Started stripping
\u0000
unicode characters when submitting new notices. PostgreSQL doesn't like it.
22.05a - 2022-05-13
- Stopped using the
infringing_urls
,copyrighted_url
andworks
database tables. Moved tojsonb
fields in thenotices
table.
22.05 - 2022-05-09
- Improved the content filter query for the notice model.
22.04j - 2022-04-28
- Fixed loading search facets for loggen-in users.
22.04i - 2022-04-28
- Made the search view load facets on demand, not on every search request.
- Updated the
rails
gem.
22.04h - 2022-04-22
- Stopped joining urls in content filters.
22.04g - 2022-04-22
- Created content filters.
22.04f - 2022-04-11
- Changed how redactions are handled on the
entity
model.
22.04e - 2022-04-11
- Removed the
DefaultNameOriginal
module.
22.04d - 2022-04-10
- Made the
entity
model create aNoticeUpdateCall
only when thename_original
attribute was set.
22.04c - 2022-04-10
- Updated the
mark_notices_to_reindex_after_relations_update
task to get more feedback.
22.04b - 2022-04-08
- Removed the
skylight
gem.
22.04a - 2022-04-07
- Stopped redacting entities zip codes.
22.04 - 2022-04-04
- Started redacting
entity
model fields.
22.03c - 2022-03-25
- Moved from using the invisible
recaptcha
(v3) to the visible one (v2).
22.03b - 2022-03-25
- Updated the
rails
gem with its dependencies.
22.03a - 2022-03-02
- Hid the
works_json
field from the edit notice form in the admin. - Optimized the
copy_urls_to_notices_as_jsonb
script. Thanks to @hasegeli.
22.03 - 2022-03-01
- Added the
case_id_number
field to the public single notice view.
22.02g - 2022-02-28
- Added a new field called
case_id_number
to thenotices
table.
22.02f - 2022-02-25
- Added
tmux
to theDocker
dev machine. - Allowed admins to create new media mentions.
22.02e - 2022-02-24
- Add
Docker
support to thedevelopment
environment.
22.02d - 2022-02-20
- Fixed a grammar issue in the captcha error message.
22.02c - 2022-02-15
- Updated the
rails
gem. - Started storing urls as
json
in theworks_json
column in thenotices
table.
22.02b - 2022-02-10
- Started using the
jsonapi-serializer
gem instead ofactive_model_serializers
.
22.02a - 2022-02-04
- Fixed searching from the single notice view.
22.02a - 2022-02-04
- Fixed searching from the single notice view.
22.02 - 2022-02-03
- Added a
notes
field to theusers
model. - Updated multiple gems.
22.01m - 2022-01-31
- Stopped using a deprecated
elasticsearch
attribute (missing
).
22.01l - 2022-01-28
- Added a catch-all route.
22.01k - 2022-01-28
- Cleaned the error log from multiple minor errors.
22.01j - 2022-01-26
- Enabled the
paranoid
mode indevise
.
22.01i - 2022-01-26
- Fixed rendering of the recent notices box on mobile devices.
- Added a submission_id field and filter to the notices list in admin.
22.01h - 2022-01-21
- Allowed the
logger
helper to use custom log file locations.
22.01g - 2022-01-21
- Started catching and logging requests with formats other than
html
andjson
.
22.01f - 2022-01-20
- Started catching and logging
ActionController::UnknownFormat
errors.
22.01e - 2022-01-20
- Started catching and logging
ActionController::RoutingError
errors.
22.01d - 2022-01-18
- Stopped using the legacy database in the YouTube import background task.
22.01c - 2022-01-17
- Created a
researcher_truncated_urls
role.
22.01b - 2022-01-13
- Started using the
zeitwerk
autoloader.
22.01a - 2022-01-11
- Made sure a related notices update call runs only when there are any model related changes.
22.01 - 2022-01-10
- Made marking related notices on work/entity/topic updates run in a rake task.
21.12e - 2021-12-19
- Disabled the API for anonymous users.
21.12d - 2021-12-19
- Fixed the path of the captcha gateway loader gif.
21.12c - 2021-12-19
- Updated the
rails
gem. - Added reCAPTCHA to the search view.
21.12b - 2021-12-19
- Added a new search filter to be able to search by entities country codes.
21.12a - 2021-12-03
- Allowed to set a custom list of notice types in the submitter widget.
21.12 - 2021-12-01
- Anonymized email addresses and ips in token urls.
21.11 - 2021-11-13
- Added a new admin action for listing notices by number of token urls requested.
21.10d - 2021-10-28
- Added the full_notice_time_limit field to the users list view in the admin.
21.10c - 2021-10-28
- Allowed to sort users by a role in the admin.
21.10b - 2021-10-25
- Corrected the submitter widget doc link in the submitter request approval email.
21.10a - 2021-10-23
- Made flash messages stop using the session in the submitter widget submission form.
21.10 - 2021-10-22
- Added the submitter JavaScript widget.
21.09n - 2021-09-30
- Allowed admins to create notices.
21.09m - 2021-09-21
- Fixed the default timezone for times in the admin.
21.09l - 2021-09-21
- Created a new view listing media mentions.
21.09k - 2021-09-14
- Changed order of email filtering when requesting token urls.
21.09j - 2021-09-13
- Started cleaning up
gmail
emails from periods when requesting token urls.
21.09i - 2021-09-12
- Made email spam filtering use pattern matching.
21.09h - 2021-09-12
- Limited requests of new token urls per IP address.
21.09g - 2021-09-10
- Started checking an IP address of a user requesting a token url against the stopforumspam database.
21.09f - 2021-09-10
- Allow to block an ip of a user trying to request a token url.
21.09e - 2021-09-10
- Downcased an email address when validating it against the spam database.
21.09d - 2021-09-10
- Moved custom, blocked token url domains from
env
to the database.
21.09c - 2021-09-09
- Made the token urls request view split an email address properly.
21.09b - 2021-09-09
- Limited spam requests of new token urls.
21.09a - 2021-09-08
- Made the search view show unredacted search results only to super admins.
21.09 - 2021-09-07
- Updated the
rails
gem.
21.08e - 2021-08-17
- Updated the
addressable
gem.
21.08d - 2021-08-16
- Stopped verifying a TLS certificate when sending emails from the
CourtOrderReporter
class.
21.08c - 2021-08-06
- Changed the urls encoding method of paperclip file uploads.
21.08b - 2021-08-06
- Started comparing full notice time limits in the EST timezone.
- Updated the SMTP configuration.
21.08a - 2021-08-05
- Extended full notice view limits to researcher role.
21.08 - 2021-08-04
- Bumped
ruby
to3.0.2
.
21.06d - 2021-06-26
- Bumped
ruby
to2.7.3
.
21.06c - 2021-06-19
- Updated the
rails
gem to6.1.x
.
21.06b - 2021-06-14
- Added a library for importing notices from Youtube.
21.06a - 2021-06-10
- Bumped
ruby
to2.6.6
.
21.06 - 2021-06-03
- Updated the
rails
gem to6.0.x
.
21.05d - 2021-05-28
- Made the
kind
field required in thefile_upload
model. - Replaced the
expire_fragment
method with a system method call during the async indexing rake task.
21.05c - 2021-05-24
- Bumped
ruby
to2.5.9
.
21.05b - 2021-05-18
- Bumped
rails
to5.2.6
.
- Fixed the advanced search toggler.
21.05a - 2021-05-17
- Updated the token url confirmation email body.
21.05 - 2021-05-17
- Allowed to set a custom token urls interval value and archive old token urls.
- Cleaned up the admin interface.
21.04c - 2021-04-28
- Allowed to set selected users to create permanent token urls for sensitive notices.
21.04b - 2021-04-21
- Allowed any role with the
can_generate_permanent_notice_token_urls
setting be able to generate permanent token urls.
21.04a - 2021-04-19
- Added basic notice viewing stats.
21.04 - 2021-04-14
- Made the notice view show all available notice entity roles.
21.02 - 2021-02-27
- Add a new functionality that allows to show sensitive notice URLs only to researchers.
20.11b - 2020-11-24
- Fix the test suite.
20.11a - 2020-11-21
- Clean up the production log.
20.11 - 2020-11-11
- Allow to customize the search index name during the
run_catchup_es_indexing
task.
20.10a - 2020-10-20
- #628 Tweaks Elasticsearch options so that high-volume searches resolve before timeout.
20.10 - 2020-10-05
- #612 Elasticsearch updated from 5.x to 7.x.
- This includes a major refactor of the Elasticsearch-handling components to promote maintainability.
- #624 Migrate phantomjs (no longer maintained) to selenium.
- #623 Migrate from Travis (stopped working) to CircleCI.
- #626 Documentation of elasticsearch upgrade process.
- #625 Patch update to rails version.
20.09 - 2020-09-02
- #621 Don't offer a 'click here to request access' option when there is nothing further to be requested
- #622 Display date_sent even when sender name is hidden
- #622 Don't display ",,," when sender/recipient address is empty
20.08a - 2020-08-25
- #620 Stop sending
url_original
with serialized URL objects
- #619 Prefill submitter and recipient information on notice creation web form when the submitting user has a linked entity
- #618 Improve communication around deep pagination
20.08 - 2020-08-03
- #617 Placeholder notice type
- #615 Fix ReindexRun metadata creation.
- #616 Fixes bug in redirect behavior after notice submission through web form.
20.07a - 2020-07-30
- #614 Counternotice is now in the list of creatable notice types at
/notices/new/
.
20.07 - 2020-07-17
- #611 Fix bug whereby people with access tokens could not download attached documents
- #610 Improve docs for new devs (thanks, @siaw23 !)
- #609 Bug in URL deconcatenation logic, which erroneously rejected URLs that contained "http" as a substring outside of the protocol indicator.
- Updated rack version.
20.05b - 2020-05-29
- Counterfeit notice type #604
- Update version of acts-as-taggable-on to improve performance
- Override acts-as-taggable-on parser to hardcode some high-allocation calls
- Update rails (patch version) to address vulns
20.05a - 2020-05-04
20.05 - 2020-05-01
20.04c - 2020-04-30
- #598 Fixes a bug introduced in .04a whereby submitting multipart form data through the API failed due to known inconsistencies between form data and JSON data representations, and unknown differences between production and test environments.
20.04b - 2020-04-17
- #596 CMS administrators can now manage header/footer links to CMS content through the CMS.
20.04a - 2020-04-07
- #595 Validate bytelength of submitted URLs more aggressively
- Also in #595, remove NoticeSubmissionInitializer and NoticeSubmissionFinalizer in favor of a more rails-y way of handling notice creation and validation, while still preserving the option of backgrounding the slow parts later
- More Skylight instrumentation to get a more granular view into notice creation slowness
20.04 - 2020-04-01
- #593 Speed up test suite (~25%)
- #586 Upgrade simple_form (4.1 -> 5; thanks dependabot)
- #594 Upgrade nokogiri, actionview
- #594 Split apart concatenated URLs in notice creation to make separate CopyrightedUrl/InfringingUrl objects
20.01b - 2020-01-31
- CMS URL promoted to the top level, so that the CMS is now serving blog and static page content
- Maintenance page is now served from a different directory
- Tests of blog and pages functionality updated to ensure same behavior in the CMS
high_voltage-pages
CSS class renamed tostatic-pages
so as not to baffle future developers
BlogEntry
andBlogEntryTopicAssignment
models and associated codeOriginalNewsId
controller- The content was migrated into the CMS in an intermediate commit, not tagged as a release.
- Rake tasks from 20.01a (no longer usable as
BlogEntry
no longer exists) high_voltage
gem and existing static pages (app/views/pages
directory)
web-console
gem, for debugging in development- Testing of RSS feed
20.01a - 2020-01-21
- We have a CMS! (Comfortable-Mexican-Sofa)
- Rake tasks to stand up the CMS and migrate blog content
20.01 - 2020-01-08
- Upgrade rails_admin from 1.4 to 2.0
- This had been blocked by the Rails upgrade
- Should result in a noticeable speedup.
19.11a2 - 2019-11-18
- Fixes bug which prevented files from being downloaded
- Updates test suite with regresssion test for this bug
19.11a - 2019-11-15
- Upgrade to Rails 5.2 (from 4.2) (!!)
19.11 - 2019-11-06
- Upgrade to ruby 2.5.5 (from 2.3.3).
- Edits text of static pages.
19.10a - 2019-10-10
- Bug in lib/rack-attack/request.rb#token which was preventing data submission.
19.10 - 2019-10-04
- Enforced previously soft limits on API token use.
- 19.09 -- something in that was breaking notice submission
19.09 - 2019-09-06
- Back-end work to support a future statistics dashboard.
19.07.a - 2019-09-04
- Allowed for a configurable list of notices which suppresss "click here to see full URLs" feature (as some placeholder notices have no URLs, so the offer is misleading)
- Updated nokogiri (critical security fix)
19.07 - 2019-07-01
- Compresses http responses
get_approximate_count
method onNotice
andInfringingUrl
(uses postgres reltuples)- Safelists logged-in users using the web interface (if they would not be throttled using the API)
- Updates numerous dependencies
- Refactors court order reporting cron job
- Only includes notices of type supporting in this cron job
- Bug with captcha validation in notice requests (pushed out before this release as a hotfix)
- Broken link in admin site
- Strips special characters from cron job filenames to suppress annoying warning emails to admins
19.05 - 2019-05-20
- Return link to documentation when API users are missing auth token (first PR by our 2019 Google Summer of Code student shubhscoder -- thanks!)
- Cache entire topic menu as a single unit rather than individually caching subunits
- Freeze more string literals
- Specify that Elasticsearch should return only the data we actually use, not the whole record (this dramatically cuts the response time of the search action)
- Switched to CalVer, which better fits our actual release pattern.
- Made separate cache keys for truncated and un-truncated versions of notice pages to ensure users see the correct data
- Fix bug whereby so that full notice view requests were validating twice, hence appearing invalid to users
- Handle special characters in filenames, which were preventing file copy during report generation
2.2.0 - 2019-05-09
- Truncate URLs:
- show only domains plus counts to anonymous users
- show full data to logged-in users
- allow anonymous users to request full access for limited personal use
- Memory profiling (superadmins only)
- Database indexes to speed up some slow queries
- Allow for elasticsearch index names to be customized via ENV
- Show 'submitted to Lumen' date in search results
- Make risk triggers work
- Add turnout back
- Some dependency updates
- Asset-pipeline-related gems no longer loaded in prod to save on memory
- Prevent deep pagination on search results, since Elasticsearch doesn't support it
- Much lower throttling limits in rack-attack, coupled with mechanism for safelisting IPs
- Increased cache time-to-live
2.1.6.2 - 2019-02-04
- Fixed bug whereby overly aggressive caching in the notice advanced search functions was causing garbage data to show up.
2.1.6.1 - 2018-12-21
apply_metadata
needs to be public, per errors in logs.
2.1.6 - 2018-12-20
- "Submitter" field on notice submissions
- VCR dependency for ElasticsearchQuery spec
- Placeholder notice for use in Google Canadian law notice responses
- Auto-redaction of work descriptions for newly added works
- Rake task for redaction of descriptions of existing works (this will be applied slowly over time to redact our existing 203M works)
- Custom message for hidden notices
- Bug preventing notices submitted via OldChill from being imported into Lumen
- Excessive instantiations on
ElasticsearchQuery#search
leading to slow notice searches and topic/notice display pages; the API for#search
has been changed to allow its consumers to reduce their data demands - Excessive db queries on
NoticesController#create
- Excessive db queries on
RailsAdmin#MainController#edit
(resulting in minor changes to admin interface) - More extensive caching of expensive fragments
- Improved code style
- Dependencies with security vulnerabilities have been upgraded
- Elasticsearch configuration has been streamlined
- Most intermittent test failures have been eliminated
- Bug whereby supporting documents submitted via the notice submission webform were miscategorized as original
- Bug whereby entity types were not being set properly via the notice submission webform
- Bug whereby admins could not delete notices
- Links to related FAQs and blog entries on notice pages
- DMCA counternotice creator
A 2.1.6 prerelease also included a feature to truncate URLs for unauthorized users, but it was [YANKED] pending further testing.
2.1.5.2 - 2018-09-28
- Direct remaining stderr to log file in hopes that cron will stop emailing admins
2.1.5.1 - 2018-09-12
- Submitter fields on government request notice creation web form
- Cache clearing cron job (no longer tries to delete nonempty directories)
2.1.5 - 2018-09-05
- Rake task for indexing added/changed notices after a given date
- Redirect for the frequently accessed
/dmca/counter512.pdf
to a URL which actually exists - Rubocop and coverage checking in build process
- PR and issue templates
- Log formats (logs now include timestamps; API keys are logged)
- Opted in to
config.active_record.raise_in_transactional_callbacks
warnings (the prior behavior of setting these to false was deprecated) - Updated caching strategy to avoid occasional race condition wherein cache keys existed at time of check but cache contents had been deleted by time of use
- Updated http to https links where possible
- Improved handling of forged POSTs to notices (thereby removing an API call error message)
- Fixed error from trying to iterate over empty tweets
- Handled Sass deprecation warnings that had been clogging the logs
- Fixed bug whereby users with a connected entity could not submit web forms despite auth token
- Minor wording updates
- Added timeout to Elasticsearch client
- Removed excess whitespace from notice URL list (which added up to surprisingly much content for notices with many URLs)
- Updated proxy caching strategy
- Updated references to "chillingeffects" to "lumen" where possible
- Switched to postgres UPSERT for infringing URLs in order to speed up notice creation when there are very large numbers of associated URLs
- Switched to
.env
rather thanbin/init-env.sh
for environment variable handling - Documented the release process
- Improved test isolation, so that the test suite is dramatically more likely to pass
- Linted for consistency with current ruby style guidelines
- Suppressed spurious
to_ary
warning
- No longer displaying received date on notices when it is the same as the sent date
- Removed public access to the notice submission form
2.1.4.1 - 2018-08-16
Hotfixes the 2.1.4 release by adding an index to Notice.created_at
. This should help with extremely low load times for the home page (Skylight asserts that the vast majority of the time is spent on an SQL query that sorts on this unindexed column).