Treeherder Bug Filer Reporter
	Description • 6 years ago
treeherder

Filed by: philringnalda [at] gmail.com

https://treeherder.mozilla.org/logviewer.html#?job_id=175234664&repo=mozilla-release

https://queue.taskcluster.net/v1/task/IUPLmYSJQGmFt9vLhGH_ww/runs/1/artifacts/public/logs/live_backing.log

	Dustin J. Mitchell [:dustin] (he/him)
	Comment 1 • 6 years ago

I'm curious why this particular file is being downloaded, and why it's over http..

	Pete Moore [:pmoore][:pete] Assignee
	Comment 2 • 6 years ago

This is indeed extremely strange.

The reference to the schema location is in the "id" property of the schema.

Here we see the lines where the task payload is validated:

https://github.com/taskcluster/generic-worker/blob/v10.7.8/main.go#L839-L841

> 	schemaLoader := gojsonschema.NewStringLoader(taskPayloadSchema())
> 	docLoader := gojsonschema.NewStringLoader(string(jsonPayload))
> 	result, err := gojsonschema.Validate(schemaLoader, docLoader)

The schema loader takes a string as input, which is the full payload schema, which is burned into the release here:

https://github.com/taskcluster/generic-worker/blob/v10.7.8/generated_windows.go#L311-L609

Looking at the spec, I see that the property should be named "$id" rather that "id". I therefore have no idea why gojsonschema, that validates the payload, would fetch content from the url. Even if it had been named correctly ("$id" instead of "id") then it is just a URI for resolving references, rather than a URL to actively fetch content from.

I think the safest thing to do will just be to delete the property entirely.

I suspect there is some magic happening in https://github.com/xeipuuv/gojsonschema that looks for strings that appear to be URLs (possibly checking for the '#' at the end) and then preemptively downloads them. This is just a hunch, because I have no other explanation for how something would find a reference to this url, and actually try to query it. Not good.

	[github robot]
	Comment 4 • 6 years ago

Commits pushed to master at https://github.com/taskcluster/generic-worker

https://github.com/taskcluster/generic-worker/commit/81ac9f3f40e7ef454484c80d41c051daaab6964f
Bug 1456357 - remove references to http://schemas.taskcluster.net/generic-worker/v1/payload.json

https://github.com/taskcluster/generic-worker/commit/6c9eec9baee55f8f9842e6d01bb4a7011214e2b4
Merge pull request #85 from taskcluster/bug1456357

Bug 1456357 - remove references to http://schemas.taskcluster.net/generic-worker/v1/payload.json

	Pete Moore [:pmoore][:pete] Assignee
	Comment 5 • 6 years ago

Released in generic-worker 10.7.11.