Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify bst shell implementation #1431

Open
wants to merge 7 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion man/bst-artifact-checkout.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST ARTIFACT CHECKOUT" "1" "2020-08-13" "" "bst artifact checkout Manual"
.TH "BST ARTIFACT CHECKOUT" "1" "2020-10-07" "" "bst artifact checkout Manual"
.SH NAME
bst\-artifact\-checkout \- Checkout contents of an artifact
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-artifact-delete.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST ARTIFACT DELETE" "1" "2020-08-13" "" "bst artifact delete Manual"
.TH "BST ARTIFACT DELETE" "1" "2020-10-07" "" "bst artifact delete Manual"
.SH NAME
bst\-artifact\-delete \- Remove artifacts from the local cache
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-artifact-list-contents.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST ARTIFACT LIST-CONTENTS" "1" "2020-08-13" "" "bst artifact list-contents Manual"
.TH "BST ARTIFACT LIST-CONTENTS" "1" "2020-10-07" "" "bst artifact list-contents Manual"
.SH NAME
bst\-artifact\-list-contents \- List the contents of an artifact
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-artifact-log.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST ARTIFACT LOG" "1" "2020-08-13" "" "bst artifact log Manual"
.TH "BST ARTIFACT LOG" "1" "2020-10-07" "" "bst artifact log Manual"
.SH NAME
bst\-artifact\-log \- Show logs of artifacts
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-artifact-pull.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST ARTIFACT PULL" "1" "2020-08-13" "" "bst artifact pull Manual"
.TH "BST ARTIFACT PULL" "1" "2020-10-07" "" "bst artifact pull Manual"
.SH NAME
bst\-artifact\-pull \- Pull a built artifact
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-artifact-push.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST ARTIFACT PUSH" "1" "2020-08-13" "" "bst artifact push Manual"
.TH "BST ARTIFACT PUSH" "1" "2020-10-07" "" "bst artifact push Manual"
.SH NAME
bst\-artifact\-push \- Push a built artifact
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-artifact-server.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST-ARTIFACT-SERVER" "1" "2020-08-13" "" "bst-artifact-server Manual"
.TH "BST-ARTIFACT-SERVER" "1" "2020-10-07" "" "bst-artifact-server Manual"
.SH NAME
bst-artifact-server \- CAS Artifact Server
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-artifact-show.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST ARTIFACT SHOW" "1" "2020-08-13" "" "bst artifact show Manual"
.TH "BST ARTIFACT SHOW" "1" "2020-10-07" "" "bst artifact show Manual"
.SH NAME
bst\-artifact\-show \- Show the cached state of artifacts
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-artifact.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST ARTIFACT" "1" "2020-08-13" "" "bst artifact Manual"
.TH "BST ARTIFACT" "1" "2020-10-07" "" "bst artifact Manual"
.SH NAME
bst\-artifact \- Manipulate cached artifacts.
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-build.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST BUILD" "1" "2020-08-13" "" "bst build Manual"
.TH "BST BUILD" "1" "2020-10-07" "" "bst build Manual"
.SH NAME
bst\-build \- Build elements in a pipeline
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-help.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST HELP" "1" "2020-08-13" "" "bst help Manual"
.TH "BST HELP" "1" "2020-10-07" "" "bst help Manual"
.SH NAME
bst\-help \- Print usage information
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-init.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST INIT" "1" "2020-08-13" "" "bst init Manual"
.TH "BST INIT" "1" "2020-10-07" "" "bst init Manual"
.SH NAME
bst\-init \- Initialize a new BuildStream project
.SH SYNOPSIS
Expand Down
6 changes: 3 additions & 3 deletions man/bst-shell.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST SHELL" "1" "2020-08-13" "" "bst shell Manual"
.TH "BST SHELL" "1" "2020-10-07" "" "bst shell Manual"
.SH NAME
bst\-shell \- Shell into an element's sandbox environment
.SH SYNOPSIS
Expand Down Expand Up @@ -36,8 +36,8 @@ Mount a file or directory into the sandbox
\fB\-\-isolate\fP
Create an isolated build sandbox
.TP
\fB\-t,\fP \-\-use\-buildtree [ask|try|always|never]
Stage a buildtree. If `always` is set, will always fail to build if a buildtree is not available. --pull and pull-buildtrees configuration is needed if trying to query for remotely cached buildtrees. [default: ask]
\fB\-t,\fP \-\-use\-buildtree [try|always|never]
Stage a buildtree. If `always` is set, will always fail to build if a buildtree is not available. --pull and pull-buildtrees configuration is needed if trying to query for remotely cached buildtrees. [default: never]
.TP
\fB\-\-pull\fP
Attempt to pull missing or incomplete artifacts
2 changes: 1 addition & 1 deletion man/bst-show.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST SHOW" "1" "2020-08-13" "" "bst show Manual"
.TH "BST SHOW" "1" "2020-10-07" "" "bst show Manual"
.SH NAME
bst\-show \- Show elements in the pipeline
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-source-checkout.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST SOURCE CHECKOUT" "1" "2020-08-13" "" "bst source checkout Manual"
.TH "BST SOURCE CHECKOUT" "1" "2020-10-07" "" "bst source checkout Manual"
.SH NAME
bst\-source\-checkout \- Checkout sources of an element
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-source-fetch.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST SOURCE FETCH" "1" "2020-08-13" "" "bst source fetch Manual"
.TH "BST SOURCE FETCH" "1" "2020-10-07" "" "bst source fetch Manual"
.SH NAME
bst\-source\-fetch \- Fetch sources in a pipeline
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-source-push.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST SOURCE PUSH" "1" "2020-08-13" "" "bst source push Manual"
.TH "BST SOURCE PUSH" "1" "2020-10-07" "" "bst source push Manual"
.SH NAME
bst\-source\-push \- Push sources in a pipeline
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-source-track.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST SOURCE TRACK" "1" "2020-08-13" "" "bst source track Manual"
.TH "BST SOURCE TRACK" "1" "2020-10-07" "" "bst source track Manual"
.SH NAME
bst\-source\-track \- Track new source references
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-source.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST SOURCE" "1" "2020-08-13" "" "bst source Manual"
.TH "BST SOURCE" "1" "2020-10-07" "" "bst source Manual"
.SH NAME
bst\-source \- Manipulate sources for an element
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-workspace-close.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST WORKSPACE CLOSE" "1" "2020-08-13" "" "bst workspace close Manual"
.TH "BST WORKSPACE CLOSE" "1" "2020-10-07" "" "bst workspace close Manual"
.SH NAME
bst\-workspace\-close \- Close workspaces
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-workspace-list.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST WORKSPACE LIST" "1" "2020-08-13" "" "bst workspace list Manual"
.TH "BST WORKSPACE LIST" "1" "2020-10-07" "" "bst workspace list Manual"
.SH NAME
bst\-workspace\-list \- List open workspaces
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-workspace-open.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST WORKSPACE OPEN" "1" "2020-08-13" "" "bst workspace open Manual"
.TH "BST WORKSPACE OPEN" "1" "2020-10-07" "" "bst workspace open Manual"
.SH NAME
bst\-workspace\-open \- Open a new workspace
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-workspace-reset.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST WORKSPACE RESET" "1" "2020-08-13" "" "bst workspace reset Manual"
.TH "BST WORKSPACE RESET" "1" "2020-10-07" "" "bst workspace reset Manual"
.SH NAME
bst\-workspace\-reset \- Reset a workspace to its original state
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst-workspace.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST WORKSPACE" "1" "2020-08-13" "" "bst workspace Manual"
.TH "BST WORKSPACE" "1" "2020-10-07" "" "bst workspace Manual"
.SH NAME
bst\-workspace \- Manipulate developer workspaces
.SH SYNOPSIS
Expand Down
2 changes: 1 addition & 1 deletion man/bst.1
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.TH "BST" "1" "2020-08-13" "" "bst Manual"
.TH "BST" "1" "2020-10-07" "" "bst Manual"
.SH NAME
bst \- Build and manipulate BuildStream projects...
.SH SYNOPSIS
Expand Down
19 changes: 12 additions & 7 deletions src/buildstream/_frontend/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -446,15 +446,16 @@ def init_project(
# if they are available in the execution context.
#
# Args:
# element_name (str): The element's full name
# element_key (tuple): The element's display key
# element (Element): The element
#
# Returns:
# (str): The formatted prompt to display in the shell
#
def shell_prompt(self, element_name, element_key):
def shell_prompt(self, element):

_, key, dim = element_key
element_name = element._get_full_name()

_, key, dim = element._get_display_key()

if self.colors:
prompt = (
Expand Down Expand Up @@ -703,10 +704,14 @@ def _handle_failure(self, element, task, failure):
if choice == "shell":
click.echo("\nDropping into an interactive shell in the failed build sandbox\n", err=True)
try:
unique_id, element_key = element
prompt = self.shell_prompt(full_name, element_key)
unique_id, _ = element
self.stream.shell(
None, _Scope.BUILD, prompt, isolate=True, usebuildtree="always", unique_id=unique_id
None,
_Scope.BUILD,
self.shell_prompt,
isolate=True,
usebuildtree="always",
unique_id=unique_id,
)
except BstError as e:
click.echo("Error while attempting to create interactive shell: {}".format(e), err=True)
Expand Down
106 changes: 11 additions & 95 deletions src/buildstream/_frontend/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -578,8 +578,8 @@ def show(app, elements, deps, except_, order, format_):
"--use-buildtree",
"-t",
"cli_buildtree",
type=click.Choice(["ask", "try", "always", "never"]),
default="ask",
type=click.Choice(["try", "always", "never"]),
default="never",
show_default=True,
help=(
"Stage a buildtree. If `always` is set, will always fail to "
Expand Down Expand Up @@ -617,116 +617,32 @@ def shell(app, element, mount, isolate, build_, cli_buildtree, pull_, command):
from ..element import _Scope
from .._project import HostMount

scope = _Scope.BUILD if build_ else _Scope.RUN
# Buildtree can only be used with build shells
if cli_buildtree != "never":
build_ = True
else:
cli_buildtree = None

# We may need to fetch dependency artifacts if we're pulling the artifact
selection = _PipelineSelection.ALL if pull_ else _PipelineSelection.NONE
use_buildtree = None
scope = _Scope.BUILD if build_ else _Scope.RUN

with app.initialized():
if not element:
element = app.project.get_default_target()
if not element:
raise AppError('Missing argument "ELEMENT".')

elements = app.stream.load_selection((element,), selection=selection, use_artifact_config=True)

# last one will be the element we want to stage, previous ones are
# elements to try and pull
element = elements[-1]
pull_dependencies = elements[:-1] if pull_ else None

element_name = element._get_full_name()
element_key = element._get_display_key()

prompt = app.shell_prompt(element_name, element_key)
mounts = [HostMount(path, host_path) for host_path, path in mount]

artifact_is_cached = element._cached()
buildtree_is_cached = element._cached_buildtree()
buildtree_exists = element._buildtree_exists()
can_attempt_pull = app.context.pull_buildtrees and pull_

if cli_buildtree in ("always", "try"):
if buildtree_is_cached:
use_buildtree = cli_buildtree
# If element is already cached, we can check the proto to see if the buildtree existed
elif artifact_is_cached:
if not buildtree_exists:
if cli_buildtree == "always":
# Exit early if it won't be possible to even fetch a buildtree with always option
raise AppError("Artifact was created without buildtree, unable to launch shell with it")
click.echo(
"WARNING: Artifact created without buildtree, shell will be loaded without it", err=True
)
elif can_attempt_pull:
use_buildtree = cli_buildtree
click.echo(
"WARNING: buildtree is not cached locally but did exist, will attempt to pull from available remotes",
err=True,
)
else:
if cli_buildtree == "always":
# Exit early if it won't be possible to perform a fetch as pull semantics aren't present
raise AppError(
"Artifact has a buildtree but it isn't cached. Can be retried with --pull and pull-buildtrees configured"
)
click.echo("WARNING: buildtree is not cached locally, shell will be loaded without it", err=True)
# If element isn't cached at all, we can't check the proto to see if it existed so can't exit early
elif can_attempt_pull:
use_buildtree = cli_buildtree
if use_buildtree == "always":
click.echo(
"WARNING: Element is not cached so buildtree status unknown, will attempt to pull from available remotes",
err=True,
)
else:
if cli_buildtree == "always":
# Exit early as there is no buildtree locally & can_attempt_pull is False
raise AppError(
"Artifact not cached locally. Can be retried with --pull and pull-buildtrees configured"
)
click.echo("WARNING: buildtree is not cached locally, shell will be loaded without it", err=True)
else:
# If the value has defaulted to ask and in non interactive mode, don't consider the buildtree, this
# being the default behaviour of the command
if app.interactive and cli_buildtree == "ask":
if buildtree_is_cached and bool(click.confirm("Do you want to use the cached buildtree?")):
use_buildtree = "always"
elif can_attempt_pull:
# If buildtree not cached, check if it's worth presenting the user a dialogue
message = None
if artifact_is_cached:
if buildtree_exists:
message = "Buildtree not cached but can be pulled if in available remotes, do you want to use it?"
else:
message = "Element is not cached so buildtree status unknown, do you want to pull and use it?"
if message:
try:
choice = click.prompt(
message, type=click.Choice(["try", "always", "never"]), err=True, show_choices=True,
)
except click.Abort:
click.echo("Aborting", err=True)
sys.exit(-1)

if choice != "never":
use_buildtree = choice

# Raise warning if the element is cached in a failed state
if use_buildtree and element._cached_failure():
click.echo("WARNING: using a buildtree from a failed build.", err=True)

try:
exitcode = app.stream.shell(
element,
scope,
prompt,
app.shell_prompt,
mounts=mounts,
isolate=isolate,
command=command,
usebuildtree=use_buildtree,
pull_dependencies=pull_dependencies,
usebuildtree=cli_buildtree,
pull_=pull_,
)
except BstError as e:
raise AppError("Error launching shell: {}".format(e), detail=e.detail) from e
Expand Down
Loading