Pete Moore [:pmoore][:pete] Assignee
	Description • 6 years ago

In generic-worker, tasks can mount artifacts from other tasks. In almost all cases[1], the task that produces the mounted artifact(s) should have completed before the task that depends on them is scheduled.

I'd therefore like to introduce the requirement that if task A mounts artifact(s) from task B, that task A must explicitly depend on task B, otherwise the worker should resolve task A with exception/malformed-payload.

----

[1] An exception to this general principle could be that a task explicitly publishes artifact(s) during its task execution, which should be mounted by another task, yet the task is only resolved some considerable time later. With this new requirement in place, the task that wishes to mount the artifacts would have to wait for the task to complete. In reality this is already the case, since there is no trivial way to schedule the task when the artifacts are available, so in practice the task dependency would already be required. As such, this exceptional case which is probably not worth optimising, however, it could be done as follows:

We could expand the schema for an individual task dependency to be either

  * a taskId (as now) _or_
  * an object:

    {
      "task": <TASK_ID>,
      "artifacts": [
        <ARTIFACT>...
      ]
    }

In this case, task A could be scheduled as soon as all of the listed artifacts of task B have been published, which may be prior to task B completing.

	Pete Moore [:pmoore][:pete] Assignee
	Comment 1 • 6 years ago

(In reply to Pete Moore [:pmoore][:pete] from comment #0)

> In this case, task A could be scheduled as soon as all of the listed
> artifacts of task B have been published, which may be prior to task B
> completing.

In other words, a task could depend on the artifacts of other tasks (currently a task can only depend on other tasks).

	[github robot]
	Comment 3 • 6 years ago

Commits pushed to master at https://github.com/taskcluster/generic-worker

https://github.com/taskcluster/generic-worker/commit/af896563274a9ea46ebd89822632c062002a8cca
Bug 1468155 - check mounted artifact content's tasks are task dependencies

https://github.com/taskcluster/generic-worker/commit/5643391bf3b4c82a86f208d6142de1719bd10cf4
Merge pull request #100 from taskcluster/bug1468155

Bug 1468155 - check mounted artifact content's tasks are task dependencies

	Pete Moore [:pmoore][:pete] Assignee
	Comment 4 • 6 years ago

Released in generic-worker 10.8.5.

	Jonas Finnemann Jensen (:jonasfj)
	Comment 5 • 6 years ago

Comment on attachment 8984989 [details] [review]
Github Pull Request for generic-worker

as set in github