Skip to content

Latest commit

 

History

History
187 lines (134 loc) · 7.53 KB

DEVELOPER_GUIDE.md

File metadata and controls

187 lines (134 loc) · 7.53 KB
Raw

Paws developer guide

Paws software components

Paws is made up of the following components:

  • paws - The package that you install from CRAN, which loads the separate paws. packages that contain the Paws code for each AWS service.

  • paws. - Separate packages that contain the Paws code for each AWS service, e.g. paws.storage contains the code for S3, etc. These are broken into categories because putting all the code into a single package would be over the CRAN package size limit.

  • paws.common - Common code for making AWS requests. All the paws. packages use this code to send requests and decode responses. paws.common is also on CRAN.

  • make.paws - Generates the R code for each AWS service; it reads in a JSON file with metadata on each service and outputs R code that communicates with that service. make.paws is not on CRAN.

Building

The paws and the paws. packages are generated by make.paws. make.paws reads in JSON metadata on each service (e.g. S3) and outputs R code. The JSON metadata comes from the official AWS SDK for JavaScript, located in vendor/aws-sdk-js.

To build paws, you use the makefile recipe make build. Building is slow.

Building is cached so building more than once will be faster. To clear the cache, e.g. if you've updated the code generator's code, run make clean.

To see all available makefile recipes, run make.

Updating Paws

Updating paws can fall under a few different scenarios:

  1. Add an enhancement or fix a bug in the request handling code

    Most enhancements and bug fixes are to the request handling code in paws.common. For example, to add a new method of authentication, it should be added in paws.common.

  2. Fix a bug in the generated code

    Every operation has some metadata, e.g. operation name, and inputs and outputs whose elements are defined in the generated code. If there's a problem in one of these, the make.paws code generator itself needs to be fixed, then the code must be re-generated.

    Re-generating paws will change many files, so when making changes to the make.paws, submit one PR for the changes to make.paws and a separate PR with the re-generated code.

  3. Update AWS services with new parameters, operations, etc.

    AWS updates the JSON metadata with new services, operations for each service (e.g. the PutObject operation for S3), and new operation parameters. This requires re-running the code generator with make build, but generally requires no other code changes.

Testing

Paws uses a few different kinds of tests:

  1. Unit tests

    paws.common and make.paws both contain extensive unit tests. You can run these from the command line with make unit. The paws.common unit tests will also be run whenever you run R CMD check, e.g. when submitting to CRAN. Whenever fixing a bug or adding an enhancement, add tests.

  2. Integration tests

    paws automatically generates integration tests for operations that take no parameters. You can run these with make integration assuming you're using a machine with AWS credentials. This will check that you can successfully make requests for most services. These are never run automatically anywhere so must be run manually.

    Note that some of these will fail for other reasons, e.g. the operation actually requires some parameters, rate limiting, the AWS account needs to be set up to work with a service, etc. We have a way to skip certain integration tests in make.paws/R/tests_config.R but haven't implemented for all failing tests.

  3. Package checks

    Before submitting to CRAN, the packages must pass R CMD check. Use devtools::check(cran = TRUE) to run R CMD check locally, and use devtools::check_rhub() to run it for other platforms. The CRAN maintainers receive the file cran_comments.md which typically states what you used to check before submitting, and we use R-Hub.

Submitting to CRAN

paws.common

When updating paws.common and no other code, you can update just the paws.common CRAN package.

  1. Update the version number and NEWS.md.
  2. Check the package and update on CRAN.
devtools::check("paws.common", cran = TRUE)
devtools::check_rhub("paws.common")
devtools::submit("paws.common")
  1. Add a new paws.common-vX.X.X release to GitHub with the info from NEWS.md.

paws

Updating paws is a multi-step process, which requires updating all the category packages and only then updating the central paws package.

  1. Re-generate paws with make build.
  2. Update the category packages. a. Update the version numbers of the category packages. b. Run R CMD check on all of the category packages with devtools::check and devtools::check_rhub and make sure all pass. This will take a while while the checks run. You don't want some to pass and some to fail because it's probably a bug that needs to be fixed. c. Complete submission of all of the category packages to CRAN with devtools::submit. This will probably be a few days while the CRAN maintainers approve. i. Only some of the categories are actually on CRAN; categories not on CRAN are those we judged too obscure to be worth uploading.
  3. Update the main paws package. a. In the main paws package (cran/paws), update the required versions of all the category packages. This has to be done separately because part of checking a package is checking that its required dependencies exist. b. Update the main paws package NEWS.md. c. Run R CMD check on the main paws package with devtools::check. d. Submit to CRAN with devtools::submit.
  4. Add a new paws-vX.X.X release on GitHub with the info from NEWS.md.

Folder structure

  • .github/workflows - Config for continuous integration in GitHub Actions.
  • cran - CRAN packages, paws and paws..
  • docs - Package documentation not part of the packages, e.g. on credentials.
  • examples - Code examples.
  • make.paws - Package for generating the paws package code (see above).
    • inst/extdata/packages.yml - The mapping from AWS service to CRAN category package.
    • R/custom/*.R - User-visible functions to be copied wholesale for a given service (e.g. S3) in the generated code.
  • paws.common - Package with common code for making AWS requests.
    • R/custom_*.R - Non-user-visible functions to be run at request time for a given service, e.g. if a service's requests need special processing.
  • paws - Deprecated temporary code generation output folder.
  • script - One off scripts with no other home.
  • vendor/aws-sdk-js - AWS SDK for JS with JSON AWS metadata.

Code style

In general we try to use the code style conventions in the R community, e.g. http://adv-r.had.co.nz/Style.html.

  • Use <- for assignment.

  • Use snake_case for all names.

  • Use verbs for function names. Use nouns for data object names, e.g. vectors.

  • Use at most 80 characters per line, excepting only very long strings that you can't break, e.g. a URL.

FAQs