阿里雲Dataworks數據集成工具實現：OTS -> Maxcompute數據同步

Step By Step

1、配置數據源 + 測試連通性

1.1 Table Store（OTS）參數獲取
1.2、AccessKey、AccessSecret獲取
阿里雲常見參數獲取位置
1.3 Dataworks數據集成配置OTS數據源
1.4 Maxcompute數據源配置

2、源數據準備及目標表創建

2.1 OTS 源數據表

圖片.png

2.2 odps數據表創建

CREATE TABLE otstoodps (id INT,name STRING);

3、創建數據集成任務(OTS數據源當前僅支持腳本模式)

3.1 腳本Sample

{
    "type": "job",
    "steps": [
        {
            "stepType": "ots",
            "parameter": {
                "datasource": "otsdemo",
                "column": [
                    {
                        "name": "id"
                    },
                    {
                        "name": "name"
                    }
                ],
                "range": {
                    "end": [
                        {
                            "type": "INF_MAX"
                        }
                    ],
                    "begin": [
                        {
                            "type": "INF_MIN"
                        }
                    ]
                },
                "table": "otsreader2"
            },
            "name": "Reader",
            "category": "reader"
        },
        {
            "stepType": "odps",
            "parameter": {
                "partition": "",
                "truncate": true,
                "datasource": "odps_first",
                "column": [
                    "id",
                    "name"
                ],
                "emptyAsNull": false,
                "table": "otstoodps"
            },
            "name": "Writer",
            "category": "writer"
        }
    ],
    "version": "2.0",
    "order": {
        "hops": [
            {
                "from": "Reader",
                "to": "Writer"
            }
        ]
    },
    "setting": {
        "errorLimit": {
            "record": ""
        },
        "speed": {
            "concurrent": 2,
            "throttle": false
        }
    }
}