[Request for Beta Testers] Destinations V2 for Redshift is open for early access!

[cynthiaxyin]

Early Access Available!

Hi Airbyte community,

Early access for Destinations V2 for Redshift is now available, bringing you:

• One-to-one mapping: Data from one stream (endpoint or table) will now create one table in the destination - gone are the days where you will need to join multiple tables to learn about your data.
• Improved error handling: Typing errors will no longer fail your sync, and are now easily visible in a new _airbyte_meta column, allowing for better tracking of inconsistencies and resolution of issues.
• Incremental data delivery in final tables: Data will be incrementally delivered in final tables for all initial snapshots. No more waiting hours to see the first rows in your destination table.

How to Enroll

We are actively seeking and iterating on feedback to refine these features prior to general availability. Here's how you can try out the beta:

1. [For Airbyte Self-Managed only] In your Workspace Settings, upgrade destination Redshift to the early access version, 0.7.4. (Airbyte Cloud will automatically upgrade. See the changelog for further version updates.)

2. Create a new Redshift destination, and enable the Destinations V2 option.
   Keep in mind that the early access version is NOT suitable for production and we may ask you to reset your data.

3. Create connections sending data to the new Redshift destination. These will automatically use Destinations V2.
   Please use a different namespace for these connections to avoid overwriting/modifying any production data.

4. When creating syncs with your new redshift destination, be sure to choose Raw data (JSON) as your Normalization Method. In the V2 version of the Redshift destination this option will be removed, but in this "hybrid" version of the destination, this option remains for the V1 versions of the sync. You will still be getting final tables with V2 syncs, but the mechanism we will use to generate them is no longer dbt normalization - it's built into the connector.

Request for Feedback

As you explore the new functionality, we ask that you pay special attention to table and column name changes, missing or incorrect column types, and the speed and cost of your syncs.

We look forward to hearing about your experience with Redshift Destinations V2. Please don't hesitate to share any feedback or questions in the comments below. You can also join us on Slack if you would like to chat directly with the team about this feature.

[4vuedemoguy]

I was tried for Redshift V2 destination on OSS setup. I got the error. Please can you verify once. I have attached Logs also.

default_workspace_logs_702_txt.txt

[cynthiaxyin]

@4vuedemoguy Please make sure you follow step 4 in the original post and select Raw data (JSON) under the Transformation tab for your connection. This will make sure only V2 logic runs; from your screenshot and log file, it looks like the old V1 dbt normalization still ran.