Go API Documentation

github.com/TykTechnologies/tyk-pump/analytics

No package summary is available.

Package

Files: 4. Third party imports: 5. Imports from organisation: 1. Tests: 0. Benchmarks: 0.

Constants

Vars

GraphSQLTableName should be defined before SQL migration is called on the GraphRecord the reason this approach is used to define the table name is due to gorm's inability to read values from the fields of the GraphRecord/AnalyticsRecord struct when it is migrating, due to that a single static value is going to be returned as TableName and it will be used as the prefix for index/relationship creation no matter the value passed to db.Table()

lastDocumentTimestamp is a map to store the last document timestamps of different Mongo Aggregators

mutex is used to prevent concurrent writes to the same key

Types

AggregateFieldList

This type doesn't have documentation.

Field name	Field type	Comment
APIKeys	`[]Counter`	No comment on field.
APIID	`[]Counter`	No comment on field.
OauthIDs	`[]Counter`	No comment on field.
Geo	`[]Counter`	No comment on field.
Tags	`[]Counter`	No comment on field.
Errors	`[]Counter`	No comment on field.
Endpoints	`[]Counter`	No comment on field.
KeyEndpoint	`map[string][]Counter`	No comment on field.
OauthEndpoint	`map[string][]Counter`	No comment on field.
APIEndpoint	`[]Counter`	No comment on field.

AnalyticsFilters

This type doesn't have documentation.

Field name	Field type	Comment
OrgsIDs	`[]string`	Filters pump data by an allow list of org_ids.
APIIDs	`[]string`	Filters pump data by an allow list of api_ids.
ResponseCodes	`[]int`	Filters pump data by an allow list of response_codes.
SkippedOrgsIDs	`[]string`	Filters pump data by a block list of org_ids.
SkippedAPIIDs	`[]string`	Filters pump data by a block list of api_ids.
SkippedResponseCodes	`[]int`	Filters pump data by a block list of response_codes.

AnalyticsRecord

AnalyticsRecord encodes the details of a request

Field name	Field type	Comment
id	`model.ObjectID`	No comment on field.
Method	`string`	No comment on field.
Host	`string`	No comment on field.
Path	`string`	No comment on field.
RawPath	`string`	No comment on field.
ContentLength	`int64`	No comment on field.
UserAgent	`string`	No comment on field.
Day	`int`	No comment on field.
Month	`time.Month`	No comment on field.
Year	`int`	No comment on field.
Hour	`int`	No comment on field.
ResponseCode	`int`	No comment on field.
APIKey	`string`	No comment on field.
TimeStamp	`time.Time`	No comment on field.
APIVersion	`string`	No comment on field.
APIName	`string`	No comment on field.
APIID	`string`	No comment on field.
OrgID	`string`	No comment on field.
OauthID	`string`	No comment on field.
RequestTime	`int64`	No comment on field.
RawRequest	`string`	No comment on field.
RawResponse	`string`	No comment on field.
IPAddress	`string`	No comment on field.
Geo	`GeoData`	No comment on field.
Network	`NetworkStats`	No comment on field.
Latency	`Latency`	No comment on field.
Tags	`[]string`	No comment on field.
Alias	`string`	No comment on field.
TrackPath	`bool`	No comment on field.
ExpireAt	`time.Time`	No comment on field.
ApiSchema	`string`	No comment on field.
GraphQLStats	`GraphQLStats`	No comment on field.
CollectionName	`string`	No comment on field.

AnalyticsRecordAggregate

This type doesn't have documentation.

Field name	Field type	Comment
id	`model.ObjectID`	No comment on field.
TimeStamp	`time.Time`	No comment on field.
OrgID	`string`	No comment on field.
TimeID	`struct { Year int Month int Day int Hour int }`	No comment on field.
APIKeys	`map[string]*Counter`	No comment on field.
Errors	`map[string]*Counter`	No comment on field.
Versions	`map[string]*Counter`	No comment on field.
APIID	`map[string]*Counter`	No comment on field.
OauthIDs	`map[string]*Counter`	No comment on field.
Geo	`map[string]*Counter`	No comment on field.
Tags	`map[string]*Counter`	No comment on field.
Endpoints	`map[string]*Counter`	No comment on field.
Lists	`AggregateFieldList`	No comment on field.
KeyEndpoint	`map[string]map[string]*Counter`	No comment on field.
OauthEndpoint	`map[string]map[string]*Counter`	No comment on field.
ApiEndpoint	`map[string]*Counter`	No comment on field.
Total	`Counter`	No comment on field.
ExpireAt	`time.Time`	No comment on field.
LastTime	`time.Time`	No comment on field.
Mixed	`bool`	No comment on field.

City

This type doesn't have documentation.

Field name	Field type	Comment
GeoNameID	`uint`	No comment on field.
Names	`map[string]string`	No comment on field.

Code

This type doesn't have documentation.

Field name	Field type	Comment
Code1x	`int`	No comment on field.
Code200	`int`	No comment on field.
Code201	`int`	No comment on field.
Code2x	`int`	No comment on field.
Code301	`int`	No comment on field.
Code302	`int`	No comment on field.
Code303	`int`	No comment on field.
Code304	`int`	No comment on field.
Code3x	`int`	No comment on field.
Code400	`int`	No comment on field.
Code401	`int`	No comment on field.
Code403	`int`	No comment on field.
Code404	`int`	No comment on field.
Code429	`int`	No comment on field.
Code4x	`int`	No comment on field.
Code500	`int`	No comment on field.
Code501	`int`	No comment on field.
Code502	`int`	No comment on field.
Code503	`int`	No comment on field.
Code504	`int`	No comment on field.
Code5x	`int`	No comment on field.

Counter

This type doesn't have documentation.

Field name	Field type	Comment
Hits	`int`	No comment on field.
Success	`int`	No comment on field.
ErrorTotal	`int`	No comment on field.
RequestTime	`float64`	No comment on field.
TotalRequestTime	`float64`	No comment on field.
Identifier	`string`	No comment on field.
HumanIdentifier	`string`	No comment on field.
LastTime	`time.Time`	No comment on field.
OpenConnections	`int64`	No comment on field.
ClosedConnections	`int64`	No comment on field.
BytesIn	`int64`	No comment on field.
BytesOut	`int64`	No comment on field.
MaxUpstreamLatency	`int64`	No comment on field.
MinUpstreamLatency	`int64`	No comment on field.
TotalUpstreamLatency	`int64`	No comment on field.
UpstreamLatency	`float64`	No comment on field.
MaxLatency	`int64`	No comment on field.
MinLatency	`int64`	No comment on field.
TotalLatency	`int64`	No comment on field.
Latency	`float64`	No comment on field.
ErrorMap	`map[string]int`	No comment on field.
ErrorList	`[]ErrorData`	No comment on field.

Country

This type doesn't have documentation.

Field name	Field type	Comment
ISOCode	`string`	No comment on field.

Dimension

This type doesn't have documentation.

Field name	Field type	Comment
Name	`string`	No comment on field.
Value	`string`	No comment on field.
Counter	`*Counter`	No comment on field.

ErrorData

This type doesn't have documentation.

Field name	Field type	Comment
Code	`string`	No comment on field.
Count	`int`	No comment on field.

GeoData

This type doesn't have documentation.

Field name	Field type	Comment
Country	`Country`	No comment on field.
City	`City`	No comment on field.
Location	`Location`	No comment on field.

GraphError

This type doesn't have documentation.

Field name	Field type	Comment
Message	`string`	No comment on field.
Path	`[]any`	No comment on field.

GraphQLOperations

This type doesn't have documentation.

Field name	Field type	Comment
type	`int`	No comment on field.

GraphQLStats

This type doesn't have documentation.

Field name	Field type	Comment
Variables	`string`	No comment on field.
RootFields	`[]string`	No comment on field.
Types	`map[string][]string`	No comment on field.
Errors	`[]GraphError`	No comment on field.
OperationType	`GraphQLOperations`	No comment on field.
HasErrors	`bool`	No comment on field.
IsGraphQL	`bool`	No comment on field.

GraphRecord

This type doesn't have documentation.

Field name	Field type	Comment
Types	`map[string][]string`	No comment on field.
AnalyticsRecord	`AnalyticsRecord`	No comment on field.
OperationType	`string`	No comment on field.
Variables	`string`	No comment on field.
RootFields	`[]string`	No comment on field.
Errors	`[]GraphError`	No comment on field.
HasErrors	`bool`	No comment on field.

GraphRecordAggregate

This type doesn't have documentation.

Field name	Field type	Comment
	`AnalyticsRecordAggregate`	No comment on field.
Types	`map[string]*Counter`	No comment on field.
Fields	`map[string]*Counter`	No comment on field.
Operation	`map[string]*Counter`	No comment on field.
RootFields	`map[string]*Counter`	No comment on field.

GraphSQLAnalyticsRecordAggregate

This type doesn't have documentation.

Field name	Field type	Comment
ID	`string`	No comment on field.
OrgID	`string`	No comment on field.
Dimension	`string`	No comment on field.
DimensionValue	`string`	No comment on field.
APIID	`string`	No comment on field.
	`Counter`	No comment on field.
	`Code`	No comment on field.
TimeStamp	`int64`	No comment on field.

Latency

This type doesn't have documentation.

Field name	Field type	Comment
Total	`int64`	No comment on field.
Upstream	`int64`	No comment on field.

Location

This type doesn't have documentation.

Field name	Field type	Comment
Latitude	`float64`	No comment on field.
Longitude	`float64`	No comment on field.
TimeZone	`string`	No comment on field.

NetworkStats

This type doesn't have documentation.

Field name	Field type	Comment
OpenConnections	`int64`	No comment on field.
ClosedConnection	`int64`	No comment on field.
BytesIn	`int64`	No comment on field.
BytesOut	`int64`	No comment on field.

SQLAnalyticsRecordAggregate

This type doesn't have documentation.

Field name	Field type	Comment
ID	`string`	No comment on field.
	`Counter`	No comment on field.
TimeStamp	`int64`	No comment on field.
OrgID	`string`	No comment on field.
Dimension	`string`	No comment on field.
DimensionValue	`string`	No comment on field.
	`Code`	No comment on field.

UptimeReportAggregate

This type doesn't have documentation.

Field name	Field type	Comment
TimeStamp	`time.Time`	No comment on field.
OrgID	`string`	No comment on field.
TimeID	`struct { Year int Month int Day int Hour int }`	No comment on field.
URL	`map[string]*Counter`	No comment on field.
Errors	`map[string]*Counter`	No comment on field.
Total	`Counter`	No comment on field.
ExpireAt	`time.Time`	No comment on field.
LastTime	`time.Time`	No comment on field.

UptimeReportAggregateSQL

This type doesn't have documentation.

Field name	Field type	Comment
ID	`string`	No comment on field.
	`Counter`	No comment on field.
TimeStamp	`int64`	No comment on field.
OrgID	`string`	No comment on field.
Dimension	`string`	No comment on field.
DimensionValue	`string`	No comment on field.
	`Code`	No comment on field.

UptimeReportData

This type doesn't have documentation.

Field name	Field type	Comment
ID	`model.ObjectID`	No comment on field.
URL	`string`	No comment on field.
RequestTime	`int64`	No comment on field.
ResponseCode	`int`	No comment on field.
TCPError	`bool`	No comment on field.
ServerError	`bool`	No comment on field.
Day	`int`	No comment on field.
Month	`time.Month`	No comment on field.
Year	`int`	No comment on field.
Hour	`int`	No comment on field.
Minute	`int`	No comment on field.
TimeStamp	`time.Time`	No comment on field.
ExpireAt	`time.Time`	No comment on field.
APIID	`string`	No comment on field.
OrgID	`string`	No comment on field.

Functions

func AggregateData

AggregateData calculates aggregated data, returns map orgID => aggregated analytics data

func AggregateGraphData

AggregateGraphData collects the graph records into a map of GraphRecordAggregate to apiID

Uses: fmt.Sprintf.

func AggregateUptimeData

Uses: strconv.Itoa, structs.Map, time.Date.

func GeoIPLookup

Uses: fmt.Errorf, net.ParseIP.

func NewGraphRecordAggregate

func OnConflictAssignments

Uses: gorm.Expr, structs.Fields.

func OnConflictUptimeAssignments

Uses: gorm.Expr, structs.Fields.

func SetlastTimestampAgggregateRecord

SetlastTimestampAgggregateRecord sets the last timestamp for the aggregate record

func TrimTag

Uses: strings.ReplaceAll, strings.TrimSpace.

func (*AnalyticsRecord) GetFieldNames

func (*AnalyticsRecord) GetGeo

func (*AnalyticsRecord) GetLineValues

Uses: strconv.FormatBool, strconv.FormatUint, strings.Join.

func (*AnalyticsRecord) GetObjectID

func (*AnalyticsRecord) IsGraphRecord

func (*AnalyticsRecord) RemoveIgnoredFields

Uses: structs.Fields.

func (*AnalyticsRecord) SetExpiry

Uses: time.Duration, time.Hour, time.Now, time.Second.

func (*AnalyticsRecord) SetObjectID

func (*AnalyticsRecord) TableName

func (*AnalyticsRecord) TimeStampFromProto

Uses: time.LoadLocation.

func (*AnalyticsRecord) TimestampToProto

TimestampToProto will process timestamps and assign them to the proto record protobuf converts all timestamps to UTC so we need to ensure that we keep the same original location, in order to do so, we store the location

Uses: timestamppb.New.

func (*AnalyticsRecord) ToGraphRecord

func (*AnalyticsRecord) TrimRawData

func (*AnalyticsRecordAggregate) AsChange

Uses: model.DBM, time.Date.

func (*AnalyticsRecordAggregate) AsTimeUpdate

Uses: model.DBM.

func (*AnalyticsRecordAggregate) Dimensions

func (*AnalyticsRecordAggregate) DiscardAggregations

DiscardAggregations this method discard the aggregations of X field specified in the aggregated pump configuration

Uses: logrus.Fields.

func (*AnalyticsRecordAggregate) GetObjectID

func (*AnalyticsRecordAggregate) SetErrorList

Uses: model.DBM, sort.SliceStable.

func (*AnalyticsRecordAggregate) SetObjectID

func (*AnalyticsRecordAggregate) TableName

func (*Code) ProcessStatusCodes

Uses: structs.New.

func (*GeoData) GetFieldNames

func (*GeoData) GetLineValues

Uses: fmt.Sprintf, sort.Strings, strconv.FormatUint.

func (*GraphRecord) GetObjectID

GetObjectID is a dummy function to satisfy the interface

func (*GraphRecord) SetObjectID

SetObjectID is a dummy function to satisfy the interface

func (*GraphRecord) TableName

TableName is used by both the sql orm and mongo driver the table name and collection name used for operations on this model the conditional return is to ensure the right value is used for both the sql and mongo operations

func (*GraphRecordAggregate) Dimensions

func (*Latency) GetFieldNames

func (*Latency) GetLineValues

Uses: strconv.FormatUint.

func (*NetworkStats) Flush

Uses: atomic.LoadInt64, atomic.StoreInt64.

func (*NetworkStats) GetFieldNames

func (*NetworkStats) GetLineValues

Uses: strconv.FormatUint.

func (*SQLAnalyticsRecordAggregate) TableName

func (*UptimeReportAggregate) Dimensions

func (*UptimeReportAggregateSQL) TableName

func (*UptimeReportData) GetObjectID

func (*UptimeReportData) SetObjectID

func (*UptimeReportData) TableName

func (AnalyticsFilters) HasFilter

func (AnalyticsFilters) ShouldFilter

func (AnalyticsRecordAggregate) New

func (UptimeReportAggregate) New

Private functions

func doHash

References: b64.StdEncoding, strings.TrimRight.

func fnLatencySetter

func getLastDocumentTimestamp

getLastDocumentTimestamp gets the last timestamp for the aggregate record

func ignoreTag

References: strings.HasPrefix.

func incrementAggregate

incrementAggregate increments the analytic record aggregate fields using the analytics record

References: hex.EncodeToString, strconv.Itoa, structs.Map.

func incrementOrSetUnit

incrementOrSetUnit is a Mini function to handle incrementing a specific counter in our object

func intInSlice

func replaceUnsupportedChars

References: fmt.Sprintf, strings.Contains, strings.Replace.

func setAggregateTimestamp

References: time.Date, time.Duration, time.Minute, time.Time.

func stringInSlice

func trimString

References: bytes.Buffer.

func generateBSONFromProperty

References: model.DBM.

func generateSetterForTime

References: model.DBM.

func getRecords

func latencySetter

References: model.DBM.

Tests

Files: 5. Third party imports: 6. Imports from organisation: 1. Tests: 32. Benchmarks: 0.

Constants

Test functions

TestAggregateData_SkipGraphRecords

References: assert.Equal, assert.False, assert.True, testing.T.

TestAggregateGraphData

References: require.Assertions, require.New, testing.T, time.Date, time.UTC.

TestAggregateGraphData_Dimension

References: require.New, time.Date, time.UTC.

TestAggregateUptimeData

References: cmp.Diff, cmp.Equal, testing.T, time.Date, time.UTC.

TestAggregate_Tags

TestAggregatedRecord_GetObjectID

References: assert.Equal, model.NewObjectID, testing.T.

TestAggregatedRecord_SetObjectID

References: assert.Equal, model.NewObjectID, testing.T.

TestAggregatedRecord_TableName

References: assert.Equal, testing.T.

TestAnalyticsRecordAggregate_AsChange

References: cmp.Diff, cmp.Equal, model.DBM, testing.T, time.Date, time.UTC.

TestAnalyticsRecordAggregate_AsTimeUpdate

References: cmp.Diff, cmp.Equal, model.DBM, testing.T, time.Date, time.UTC.

TestAnalyticsRecordAggregate_generateBSONFromProperty

References: cmp.Diff, cmp.Equal, model.DBM, testing.T, time.Date, time.UTC.

TestAnalyticsRecordAggregate_generateSetterForTime

References: cmp.Diff, cmp.Equal, model.DBM, testing.T.

TestAnalyticsRecordAggregate_latencySetter

References: cmp.Diff, cmp.Equal, model.DBM, testing.T.

TestAnalyticsRecord_Base

References: assert.Equal, model.NewObjectID.

TestAnalyticsRecord_GetFieldNames

References: assert.Contains, assert.Equal.

TestAnalyticsRecord_GetLineValues

References: assert.Contains, assert.Equal, fmt.Sprint, structs.Fields, time.Now.

TestAnalyticsRecord_IsGraphRecord

References: assert.False, assert.True, testing.T.

TestAnalyticsRecord_RemoveIgnoredFields

References: assert.Equal, testing.T.

TestAnalyticsRecord_ToGraphRecordNew

References: base64.StdEncoding, cmp.Diff, cmpopts.IgnoreFields, testing.T, time.Date, time.UTC.

TestCode_ProcessStatusCodes

References: assert.Equal.

TestHasFilter

TestOnConflictUptimeAssignments

References: assert.Equal, clause.Expr.

TestSQLAnalyticsRecordAggregate_TableName

References: assert.Equal, testing.T.

TestSetAggregateTimestamp

References: assert.Equal, testing.T, time.Date, time.Minute, time.Now, time.Time.

TestShouldFilter

References: assert.Equal, testing.T.

TestTrimTag

References: assert.Equal.

TestUptimeReportAggregateSQL_TableName

References: assert.Equal, testing.T.

TestUptimeReportAggregate_Dimensions

References: assert.Equal, testing.T.

TestUptimeReportAggregate_New

References: assert.Equal, testing.T.

TestUptimeReportData_GetObjectID

References: assert.Equal, model.NewObjectID, testing.T.

TestUptimeReportData_SetObjectID

References: assert.Equal, model.NewObjectID, testing.T.

TestUptimeReportData_TableName

References: assert.Equal, testing.T.