Skip to content

Add COPY command support for Parquet and ORC#150

Merged
jklukas merged 5 commits into
sqlalchemy-redshift:masterfrom
dargueta:copy-parquet
Nov 29, 2018
Merged

Add COPY command support for Parquet and ORC#150
jklukas merged 5 commits into
sqlalchemy-redshift:masterfrom
dargueta:copy-parquet

Conversation

@dargueta

@dargueta dargueta commented Aug 3, 2018

Copy link
Copy Markdown
Contributor

Closes #151

  • Allow choosing Parquet and ORC as load formats (see here).
  • Allow choosing fixed_width as a load format as well for consistency with the others. Enforce the presence of the field widths argument if Formats.fixed_width is selected.

Todos

  • MIT compatible
  • Tests
  • Documentation
  • Updated CHANGES.rst

@dargueta dargueta changed the title Add COPY command support for Parquet and ORC Add COPY command support for Parquet, ORC, and Snappy Aug 4, 2018
@dargueta dargueta changed the title Add COPY command support for Parquet, ORC, and Snappy Add COPY command support for Parquet and ORC Aug 4, 2018
@dargueta

dargueta commented Sep 4, 2018

Copy link
Copy Markdown
Contributor Author

@graingert or @jklukas you look like the main contributors here. Sorry to be a pain but I just wanted to make sure someone saw this. Thanks!

@graingert

Copy link
Copy Markdown
Member

Hey there, can you split your Travis/tox changes into another PR, thanks!

@dargueta

dargueta commented Sep 4, 2018

Copy link
Copy Markdown
Contributor Author

can you split your Travis/tox changes into another PR, thanks!

Done!

@graingert

Copy link
Copy Markdown
Member

LGTM, can you clean up your commits with git rebase?

@dargueta

dargueta commented Sep 4, 2018

Copy link
Copy Markdown
Contributor Author

There was a bug in one of my tests I had to fix after your LGTM. Do you mean I should squash the commits, or do git rebase origin/master?

Edit: For some reason I'm suddenly getting this:

dargueta@tux ~/dev/sqlalchemy-redshift (copy-parquet)*$ git fetch upstream
dargueta@tux ~/dev/sqlalchemy-redshift (copy-parquet)*$ git fetch origin
dargueta@tux ~/dev/sqlalchemy-redshift (copy-parquet)*$ git checkout master
error: pathspec 'master' did not match any file(s) known to git.

This wasn't an issue earlier today, and now I can't rebase off of master with this. Ideas?

@graingert

Copy link
Copy Markdown
Member

@dargueta I think you want git pull --rebase=interactive upstream master

@dargueta

Copy link
Copy Markdown
Contributor Author

Okay, cleaned them up now.

Comment thread tests/test_copy_command.py Outdated
@dargueta

Copy link
Copy Markdown
Contributor Author

I think I've got everything fixed. Any other thoughts?

@dargueta

Copy link
Copy Markdown
Contributor Author

*bump*

@jklukas jklukas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

@jklukas jklukas merged commit 9fc8d93 into sqlalchemy-redshift:master Nov 29, 2018
@dargueta dargueta deleted the copy-parquet branch November 29, 2018 22:36
@jklukas

jklukas commented Dec 11, 2018

Copy link
Copy Markdown
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add support for Parquet and ORC formats to COPY

3 participants