package jdapi import ( "crypto/md5" "encoding/json" "errors" "fmt" "io/ioutil" "net" "net/http" "net/url" "sort" "strconv" "strings" "time" "git.rosy.net.cn/baseapi/utils" "github.com/satori/go.uuid" "go.uber.org/zap" ) const ( // JDErrorCodeSuccess 操作成功 JDErrorCodeSuccess = "0" // JDerrorCodeAccessFailed 操作失败,请重试,如还不能正常提供服务,可以提交工单(https://opengd.jd.com)详细说明一下,开放平台跟进回复。 JDerrorCodeAccessFailed = "-1" // JDerrorCodeFailedCanAutoRetry 操作失败,系统会自动重试,如还不能正常提供服务,可以提交工单(https://opengd.jd.com)详细说明一下,开放平台跟进回复。 JDerrorCodeFailedCanAutoRetry = "-10000" JDerrorCodeFailedAccessDB = "10000" // JDerrorCodeMissingMandatoryParam 请求参数中缺失必填项信息,请检查,如不清楚,请访问到家开放平台(https://opendj.jd.com)API分组有接口详细说明。 JDerrorCodeMissingMandatoryParam = "10005" JDerrorCodeInvalidToken = "10013" JDerrorCodeInvalidSign = "10014" JDerrorCodeAbnormalParam = "10015" JDerrorCodeMethodNotExist = "10018" JDErrorCodeExceedLimit = "10032" JDErrorCodeTimeout = "100022" JDErrorCodeTomcateFailed = "100023" JDErrorCodeLoadUnexpected = "100024" ) type JDAPI struct { token string appKey string appSecret string logger *zap.Logger sugarLogger *zap.SugaredLogger client http.Client } const ( sleepSecondWhenLimited = 6 * time.Second maxRetryCountWhenNetworkException = 3 maxRetryCountWhenReachLimited = 10 jdAPIURL = "https://openo2o.jd.com/djapi/%s" AllPage = 0 DefaultPageSize = 50 ) var ( ErrJdParam = errors.New("can not marshal jd param") ErrSystemErrMaxRetry = errors.New("JD System error reach max retry count!") ErrLimitReachMaxRetry = errors.New("JD Reach max retry count!") ErrHttpCode = errors.New("JD HTTP Code is not 200") ErrFormatWrong = errors.New("JD Result format is strange!") ErrJDCode = errors.New("JD code is not 0") ErrInnerCodeIsNotOk = errors.New("JD result inner code is not ok") exceedLimitCodes = map[string]int{ JDErrorCodeExceedLimit: 1, } // todo replace all magic number canRetryCodes = map[string]int{ JDerrorCodeFailedCanAutoRetry: 1, JDerrorCodeAccessFailed: 1, JDerrorCodeFailedAccessDB: 1, JDErrorCodeTimeout: 1, JDErrorCodeTomcateFailed: 1, JDErrorCodeLoadUnexpected: 1, } jdResultInnerCodeKeys = []string{"code", "retCode", "errorCode"} jdResultInnerCodeOK = map[string]int{ "None": 1, "0": 1, "1": 1, "190005": 1, // 部分失败 } jdResultNoPageInnerDataKeys = []string{"result", "data"} jdResultPageInner2DataKeys = []string{"result", "resultList"} jdResultPageTotalCountKeys = []string{"totalCount", "count"} ) type PageResultParser func(map[string]interface{}, int) ([]interface{}, int) func getUUID() string { return strings.ToUpper(strings.Replace(uuid.Must(uuid.NewV1()).String(), "-", "", -1)) } func getJDOperator() string { return time.Now().Format("2006-01-02_15:04:05") } func getCurTimeStr() string { return time.Now().Format("2006-01-02 15:04:05") } func SignParams(jdParams map[string]string) string { var keys []string for k := range jdParams { if k != "app_secret" { keys = append(keys, k) } } sort.Strings(keys) allStr := "" for _, k := range keys { allStr += k + jdParams[k] } allStr = jdParams["app_secret"] + allStr + jdParams["app_secret"] return fmt.Sprintf("%X", md5.Sum([]byte(allStr))) } func genGetURL(baseURL, apiStr string, params map[string]string) string { fullURL := "" if params != nil { for k, v := range params { if fullURL == "" { fullURL = "?" } else { fullURL += "&" } fullURL += k + "=" + url.QueryEscape(v) } } return fmt.Sprintf(baseURL, apiStr) + fullURL } func NewJDAPI(token, appKey, appSecret string, logger *zap.Logger) *JDAPI { return &JDAPI{token, appKey, appSecret, logger, logger.Sugar(), http.Client{Timeout: time.Second * 10}} } func (j JDAPI) AccessJDQuery(apiStr string, jdParams map[string]string) (map[string]interface{}, error) { params := make(map[string]string) params["v"] = "1.0" params["format"] = "json" params["app_key"] = j.appKey params["app_secret"] = j.appSecret params["token"] = j.token if jdParams == nil { jdParams = make(map[string]string, 0) } jdParamStr, err := json.Marshal(jdParams) if err == nil { params["jd_param_json"] = string(jdParamStr) } else { return nil, ErrJdParam } params["timestamp"] = getCurTimeStr() sign := SignParams(params) params["sign"] = sign exceedLimitRetryCount := 0 systemErrorRetryCount := 0 for { fullURL := genGetURL(jdAPIURL, apiStr, params) j.sugarLogger.Debugf("fullURL:%v", fullURL) response, err := j.client.Get(fullURL) if err != nil { j.sugarLogger.Debugf("client.Get return err:%v", err) err, ok := err.(net.Error) systemErrorRetryCount++ if ok && err.Timeout() && systemErrorRetryCount <= maxRetryCountWhenNetworkException { continue } else { return nil, err } } defer response.Body.Close() if response.StatusCode != 200 { j.sugarLogger.Debugf("http code is:%d", response.StatusCode) systemErrorRetryCount++ if systemErrorRetryCount <= maxRetryCountWhenNetworkException { continue } return nil, ErrHttpCode } var jsonResult map[string]interface{} bodyData, err := ioutil.ReadAll(response.Body) if err != nil { j.sugarLogger.Debugf("ioutil.ReadAll return:%v", err) return nil, err } err = utils.UnmarshalUseNumber(bodyData, &jsonResult) if err != nil { j.sugarLogger.Debugf("json.Unmarshal return:%v", err) return nil, err } code := jsonResult["code"].(string) if code == "0" { return jsonResult, nil } j.sugarLogger.Debugf("jd code is:%s", code) if _, ok := exceedLimitCodes[code]; ok { exceedLimitRetryCount++ if exceedLimitRetryCount <= maxRetryCountWhenReachLimited { time.Sleep(sleepSecondWhenLimited) } else { return jsonResult, ErrLimitReachMaxRetry } } else if _, ok := canRetryCodes[code]; ok { systemErrorRetryCount++ if systemErrorRetryCount > maxRetryCountWhenNetworkException { return jsonResult, ErrSystemErrMaxRetry } } else { return jsonResult, ErrJDCode } } } func (j JDAPI) AccessJDQueryNoPage(apiStr string, jdParams map[string]string, keyToRemove, keyToKeep []string) (interface{}, error) { jsonResult, err := j.AccessJDQuery(apiStr, jdParams) if err != nil { return jsonResult, err } var data map[string]interface{} err = utils.UnmarshalUseNumber([]byte(jsonResult["data"].(string)), &data) if err != nil { return jsonResult, err } innerCode := "" for _, innerCodeKey := range jdResultInnerCodeKeys { if innerCode2, ok := data[innerCodeKey]; ok { innerCode = innerCode2.(string) break } } if _, ok := jdResultInnerCodeOK[innerCode]; ok { for _, innerDataKey := range jdResultNoPageInnerDataKeys { if innerData, ok := data[innerDataKey]; ok { return utils.DictKeysMan(innerData, keyToRemove, keyToKeep), nil } } panic("Can not find inner data") } else { return jsonResult, ErrInnerCodeIsNotOk } } func NormalJDQueryHavePageResultParser(data map[string]interface{}, totalCount int) ([]interface{}, int) { var result map[string]interface{} var retVal []interface{} tempResult := data["result"] if resultStr, ok := tempResult.(string); ok { if err := utils.UnmarshalUseNumber([]byte(resultStr), &tempResult); err != nil { panic("Wrong format") } } result = tempResult.(map[string]interface{}) if totalCount == 0 { for _, totalCountKey := range jdResultPageTotalCountKeys { if totalCount2, ok := result[totalCountKey]; ok { totalCountInt64, _ := totalCount2.(json.Number).Int64() totalCount = int(totalCountInt64) if totalCount == 0 { return make([]interface{}, 0), 0 } break } } if totalCount == 0 { panic("can not find totalCount key") } } for _, inner2ResultKey := range jdResultPageInner2DataKeys { if inner2Result, ok := result[inner2ResultKey]; ok { if inner2ResultStr, ok := inner2Result.(string); ok { err := utils.UnmarshalUseNumber([]byte(inner2ResultStr), &retVal) if err != nil { panic("can not unmarshal inner2result") } } else { retVal = inner2Result.([]interface{}) } return retVal, totalCount } } panic("wrong format") } func (j JDAPI) AccessJDQueryHavePage(apiStr string, jdParams map[string]string, keyToRemove, keyToKeep []string, pageResultParser PageResultParser) ([]interface{}, error) { if pageResultParser == nil { pageResultParser = NormalJDQueryHavePageResultParser } localJdParams := make(map[string]string) if jdParams != nil && len(jdParams) > 0 { for k, v := range jdParams { localJdParams[k] = v } } totalCount := 0 pageNo := AllPage pageSize := DefaultPageSize if tempPageNo, ok := localJdParams["pageNo"]; ok { pageNo, _ = strconv.Atoi(tempPageNo) } if tempPageSize, ok := localJdParams["pageSize"]; ok { pageSize, _ = strconv.Atoi(tempPageSize) } curPage := pageNo if curPage == AllPage { curPage = 1 } retVal := make([]interface{}, 0) for { localJdParams["pageNo"] = strconv.FormatInt(int64(curPage), 10) localJdParams["pageSize"] = strconv.FormatInt(int64(pageSize), 10) jsonResult, err := j.AccessJDQuery(apiStr, localJdParams) if err != nil { return nil, err } var data map[string]interface{} err = utils.UnmarshalUseNumber([]byte(jsonResult["data"].(string)), &data) if err != nil { return nil, err } innerCode := data["code"].(string) if innerCode != "0" && innerCode != "200" { return nil, ErrInnerCodeIsNotOk } inResult, totalCount2 := pageResultParser(data, totalCount) totalCount = totalCount2 inResult = utils.DictKeysMan(inResult, keyToRemove, keyToKeep).([]interface{}) retVal = append(retVal, inResult...) if curPage == pageNo || curPage*pageSize >= totalCount { break } curPage++ } return retVal, nil }