大數據

DataWorks百問百答50:MongoDB的Object字段如何同步到ODPS?

數據

original字段為Object,下面是MongoDB裡的數據,應該如何同步至ODPS?

{
  "_id" : ObjectId("yyyyy"),
  "tags" : [ ],
  "status" : "delete",
  "category" : "xx",
  "chainId" : "xxxxx",
  "filename" : "xxxxx",
  "name" : "xxxx",
  "createdTime" : ISODate("xxxxxx"),
  "fileType" : "xxxx",
  "bucket" : "xxxx",
  "appId" : "xxxxx",
  "path" : "2018/07/18",
  "directory" : "r8xdcsdfdwH",
  "creator" : "XcdcdscdRF",
  "modifiedTime" : ISODate("2018-07-18T08:10:37.2.cdcdcd"),
  "sort" : cqfcerferf,
  "original" : {
    "creator" : "54cced",
    "modifiedTime" : ISODate("2018-07-16T11:01:19.cedcwedweZ")
  },
  "id" : "7cwecwecea",
  "extendData" : {

  },
  "fileSize" : 4235345440,
  "meta" : {
    "ImageHeight" : "454325345",
    "ImageWidth" : "452345348"
  }
}

配置json案例:

注意:不能用大寫只識別小寫的配置

  • DOCUMENT.STRING改為document.string
  • DOCUMENT.DATE改為document.date
{
    "type": "job",
    "steps": [
        {
            "stepType": "mongodb",
            "parameter": {
                "datasource": "數據源名",
                "column": [
                 
                    {
                        "name": "staddswds",
                        "type": "string"
                    },
                    {
                        "name": "sorsdst",
                        "type": "int"
                    },
                    {
                        "name": "original.creator",
                        "type": "document.string"//注意小寫
                    },
                    {
                        "name": "original.modifiedTime",
                        "type": "document.date"//注意小寫
                    }
                ],
                "collectionName": "files"
            },
            "name": "Reader",
            "category": "reader"
        },
        {
            "stepType": "odps",
            "parameter": {
                "partition": "ds=${bizdate}",
                "truncate": true,
                "compress": false,
                "datasource": "數據源名",
                "column": [
                    "staddswds",
                    "sorsdst",
                    "original_creator",
                    "original_modifiedTime"
                ],
                "emptyAsNull": false,
                "guid": "odps.sssssdf",
                "table": "表名"
            },
            "name": "Writer",
            "category": "writer"
        }
    ],
    "version": "2.0",
    "order": {
        "hops": [
            {
                "from": "Reader",
                "to": "Writer"
            }
        ]
    },
    "setting": {
        "jvmOption": "",
        "errorLimit": {
            "record": "0"
        },
        "speed": {
            "throttle": false,
            "concurrent": 3
        }
    }
}

DataWorks百問百答歷史記錄 請點擊這裡查看>>

更多DataWorks技術和產品信息,歡迎加入【DataWorks釘釘交流群】

Leave a Reply

Your email address will not be published. Required fields are marked *