Add mixed-mode stepping with a focus-set policy (new default)

CHANGELOG.md

@@ -2,9 +2,33 @@
 
 ## Version 0.9.0 (unreleased)
 
-This release overhauls the terminal UI. The most visible changes: the debugger
-now runs full screen by default, `bp`/`f`/`w` open interactive menus, and a new
-`debug>` REPL mode is entered with `)`.
+This release adds mixed-mode stepping (the new default) and overhauls the
+terminal UI. The most visible changes: stepping over Base/dependency code runs
+at native speed, the debugger runs full screen by default, `bp`/`f`/`w`/`focus`
+open interactive menus, and a new `debug>` REPL mode is entered with `)`.
+
+### Mixed-mode stepping — the new default
+
+Instead of interpreting everything (slow) or stepping over compiled code
+blindly (breakpoints missed), the debugger now interprets only calls that can
+reach the "focus set" — `Main`, packages loaded from dev checkouts, the package
+you `@enter`ed, and anything you add — and runs everything else natively.
+Callbacks and user types passed into Base or dependency code keep the carrying
+call chain interpreted, so `map(f, xs)`, `sort(xs; by = f)`, broadcasts and
+do-blocks remain fully debuggable while e.g. `sort(::Vector{Int})` runs at
+compiled speed. Breakpoints anywhere outside the focus set automatically fall
+back to full interpretation, with a message.
+
+- The prompt shows the mode: `1|debug>` (mixed, the default),
+  `1|interpreted>`, `1|compiled>` — each with its own color.
+- `M` toggles mixed ↔ interpreted; `C` still toggles compiled mode.
+- `mode [interpreted|mixed|compiled]` shows/sets the mode;
+  `Debugger.set_default_mode!(:interpreted)` (e.g. in `startup.jl`) restores
+  the previous default.
+- `focus` shows and manages the focus set; stepping into a dependency package
+  adds it to the set for the session (a message is printed).
+- API: `Debugger.interpret_module!`, `Debugger.compile_module!`,
+  `Debugger.reset_module!`, `Debugger.interpreted_modules`.
 
 ### Keys at the `debug>` prompt
 
@@ -14,6 +38,7 @@ Pressed at the beginning of an empty prompt line:
 |:----|:-------|
 | `` ` `` | enter evaluation mode (as before) |
 | `C` | toggle compiled mode (as before) |
+| `M` | **new** — toggle mixed mode |
 | `L` | toggle lowered code (as before) |
 | `T` | **new** — cycle how variable types are shown: compact → none → types only → full |
 | `S` | **new** — toggle "sticky" (full-screen) mode |
@@ -36,6 +61,8 @@ scrolling transcript back.
   are toggleable rows in the same menu.
 - `f` (no arguments) opens a frame picker for the call stack.
 - `w` (no arguments) opens the watch list (**`d`** deletes an entry).
+- `focus` (no arguments) opens the mixed-mode focus set: **space/enter**
+  toggles whether a module is interpreted, **`a`** adds one by name.
 
 All existing subcommands (`bp rm 2`, `bp add foo:12`, `f 3`, `w rm 1`, ...)
 still work, and non-interactive terminals automatically get the old text

Project.toml

@@ -1,6 +1,6 @@
 name = "Debugger"
 uuid = "31a5f54b-26ea-5ae9-a837-f05ce5417438"
-version = "0.8.1"
+version = "0.9.0"
 
 [deps]
 CodeTracking = "da1fd8a2-8d9e-5ec2-8556-3022fb5608a2"
@@ -18,7 +18,7 @@ TerminalRegressionTests = {url = "https://github.com/JuliaDebug/TerminalRegressi
 [compat]
 CodeTracking = "2, 3"
 Highlights = "0.6"
-JuliaInterpreter = "0.10, 0.11"
+JuliaInterpreter = "0.11"
 PrecompileTools = "1"
 TerminalRegressionTests = "0.3"
 julia = "1.10"

README.md

@@ -66,6 +66,8 @@ Misc:
 - `o`: open the current line in an editor
 - `q`: quit the debugger, returning `nothing`
 - `C`: toggle compiled mode
+- `M`: toggle mixed mode (see "Stepping modes" below)
+- `mode [interpreted|mixed|compiled]`: show or set the stepping mode
 - `L`: toggle showing lowered code instead of source code
 - `T`: cycle how variable types are shown: compact (long type parameters elided), no types, types only, full types
 - `S`: toggle "sticky" (full-screen) mode, on by default: the debugger runs on the terminal's alternate screen (restored when quitting, like `less` or `vim`) and redraws the status in place instead of scrolling
@@ -232,28 +234,74 @@ Hit breakpoint:
 >[1] f(x) at REPL[6]:3
 ```
 
-### Compiled mode
+### Stepping modes: interpreted, mixed and compiled
 
-In order to fully support breakpoints, the debugger interprets all code, even code that is stepped over.
-Currently, there are cases where the interpreter is too slow for this to be feasible.
-A workaround is to use "compiled mode" which is toggled by pressing `C` in the debug REPL mode (note the change of prompt color).
-When using compiled mode, code that is stepped over will be executed
-by the normal julia compiler and run just as fast as normally.
-The drawback is that breakpoints in compiled code that is stepped over are missed.
+In order to fully support breakpoints, the debugger can interpret all code, even code that is stepped
+over. Currently, there are cases where the interpreter is too slow for this to be feasible. The
+debugger therefore supports three stepping modes:
 
-To split the difference between these two extremes, one can fine tune which modules are compiled and which are not.
-For example, to compile all code in Base, even when not in `C` mode, and hence break on all specified points in user code,
-issue the following commands on the REPL *before* `@enter`:
+- **mixed** (the default): calls into modules outside the "focus set" — `Core`, `Base`, the standard
+  libraries *and* dependency packages — run natively, at full compiled speed, *unless* the call could
+  reach focus code. A call is interpreted whenever the callee or any argument carries a function or
+  type from a focus module, also inside wrappers: `map(f, xs)`, `sort(xs; by = f)`, `f.(xs)`,
+  do-blocks, `Vector{MyType}` arguments and so on all remain fully debuggable, while e.g.
+  `sort(::Vector{Int})` or a `DataFrames` call operating purely on its own types runs at native
+  speed. As the default, it keeps the plain `1|debug>` prompt.
+- **interpreted**: everything is interpreted; breakpoints always work, including the corner cases
+  mixed mode cannot catch (see below). Toggled with the `M` key (note the `1|interpreted>` prompt
+  and its color).
+- **compiled**: everything that is stepped over runs natively. Fastest, but breakpoints in code that
+  is stepped over are missed. Toggled with the `C` key (note the `1|compiled>` prompt and its color).
+
+The mode can also be shown/set with the `mode [interpreted|mixed|compiled]` command in the debug REPL,
+and the mode new sessions start in can be set (e.g. in `startup.jl`) with:
 
 ```jl
-using JuliaInterpreter, MethodAnalysis
-union!(JuliaInterpreter.compiled_modules, child_modules(Base))
+Debugger.set_default_mode!(:interpreted)
 ```
 
-Additional imported modules can also always be compiled with:
+The focus set — the modules mixed mode interprets — is seeded automatically at each `@enter`/`@run`:
+
+- `Main` (functions defined at the REPL or in scripts);
+- every loaded package whose source is a *dev checkout* (a `pkgdir` outside the read-only
+  `~/.julia/packages` store), i.e. the packages you are plausibly developing;
+- the package of the function you entered (so `@enter DataFrames.groupby(df, :a)` makes `DataFrames`
+  debuggable even though it is a registered install) — except `Base`/stdlibs, which would de-facto
+  disable the fast path;
+- stepping into (`s`) a frame of a dependency package adds it to the focus set for the session (an
+  info message is printed). Stepping into `Base`/a stdlib does *not* — a hint is printed instead.
+
+Inside the debugger, `focus` opens an interactive menu over the current set (space/enter toggles a
+module, `a` adds one by name); `focus add SomeMod` / `focus rm SomeMod` (or `focus rm i` by number)
+do the same non-interactively. These adjust the session; the permanent equivalents, e.g. for
+`startup.jl`:
 
 ```jl
-union!(JuliaInterpreter.compiled_modules, SomePackage)
+Debugger.interpret_module!(SparseArrays) # always interpret (debug into) this module
+Debugger.compile_module!(MyBigHelperPkg) # never interpret this module (also skips its `@bp` markers)
+Debugger.interpreted_modules()           # show the current focus set
+```
+
+For soundness, mixed mode automatically falls back to full interpretation while any breakpoint is set
+outside the focus set (including file breakpoints that could match such a module), while a breakpoint
+is set in a focus package that loaded non-focus packages depend on (native code in the latter could
+otherwise run past it), or while `break_on(:error)` or `break_on(:throw)` is active — a message is
+printed when this happens. Known limitations of mixed mode (switch to interpreted mode if they
+matter): focus-module callbacks hidden in type-erased containers (e.g. `Vector{Any}`, fields typed
+`::Function`) are not detected; neither are focus-module methods whose signatures only mention
+non-focus types (type piracy), nor focus code reached without any focus-typed argument at the call
+site (callbacks stored in global registries, `Task`s, finalizers). `@bp` markers inside non-focus
+modules are skipped, and file breakpoints on source that was `eval`'d or `include`d from outside a
+package directory may not trigger the fallback.
+
+The [`JuliaInterpreter.compiled_methods`/`compiled_modules`](https://github.com/JuliaDebug/JuliaInterpreter.jl)
+mechanisms keep working in every mode and take precedence over mixed mode's decisions
+(as does `JuliaInterpreter.interpreted_methods`, in the other direction). For example, to compile all
+code in Base even in interpreted mode:
+
+```jl
+using JuliaInterpreter, MethodAnalysis
+union!(JuliaInterpreter.compiled_modules, child_modules(Base))
 ```
 
 
@@ -270,7 +318,7 @@ shows the current settings. The available options are:
 - `max_vars::Int`: maximum number of variables shown in the automatic status display (default: `15`)
 - `sticky::Bool`: "full screen" mode — the debugger runs on the terminal's alternate screen and redraws the status in place instead of scrolling (default: `true`); can also be toggled with the `S` key
 - `charset::Symbol`: `:unicode` or `:ascii` (default: `:unicode`)
-- `menus::Bool`: use the interactive menus for `bp`, `f` and `w` (default: `true`)
+- `menus::Bool`: use the interactive menus for `bp`, `f`, `w` and `focus` (default: `true`)
 
 The older entry points `Debugger.set_theme(theme)` and `Debugger.set_highlight(false)` still work.
 

src/Debugger.jl

@@ -21,6 +21,7 @@ const SEARCH_PATH = []
 function __init__()
     append!(SEARCH_PATH,[joinpath(Sys.BINDIR,"../share/julia/base/"),
             joinpath(Sys.BINDIR,"../include/")])
+    init_mixed_mode()
     auto_install_repl_mode()
     return nothing
 end
@@ -38,13 +39,17 @@ using .LineNumbers: SourceFile, compute_line
 # watch expressions between invocations of the debugger interface
 const WATCH_LIST = []
 
+include("mixed_mode.jl")
+
 Base.@kwdef mutable struct DebuggerState
     frame::Union{Nothing, Frame}
     level::Int = 1
     broke_on_error::Bool = false
     watch_list::Vector = WATCH_LIST
     lowered_status::Bool = false
-    interp::Interpreter = RecursiveInterpreter()
+    interp::Interpreter = interp_for_mode(DEFAULT_MODE[])
+    # the mode to come back to when the `C` (compiled mode) toggle is switched off
+    noncompiled_interp::Interpreter = RecursiveInterpreter()
     repl = nothing
     terminal = nothing
     main_mode = nothing
@@ -65,10 +70,44 @@ function output_stream(state::DebuggerState)
     return IOContext(io, :color => have_color)
 end
 
+# `C` key: binary toggle between compiled mode and the previous non-compiled mode
 function toggle_mode(state)
-    state.interp = (state.interp === RecursiveInterpreter() ? (state.interp = NonRecursiveInterpreter()) : (state.interp = RecursiveInterpreter()))
+    if state.interp isa NonRecursiveInterpreter
+        state.interp = state.noncompiled_interp
+    else
+        state.noncompiled_interp = state.interp
+        state.interp = NonRecursiveInterpreter()
+    end
+end
+
+# `M` key: binary toggle between mixed and interpreted mode
+function toggle_mixed(state)
+    if state.interp isa MixedInterpreter
+        state.interp = RecursiveInterpreter()
+    else
+        state.interp = MixedInterpreter()
+    end
+    state.noncompiled_interp = state.interp
 end
 
+function set_session_mode!(state, mode::Symbol)
+    new_interp = interp_for_mode(mode)
+    if new_interp isa NonRecursiveInterpreter
+        # remember the current mode so the `C` toggle returns to it
+        if !(state.interp isa NonRecursiveInterpreter)
+            state.noncompiled_interp = state.interp
+        end
+    else
+        state.noncompiled_interp = new_interp
+    end
+    state.interp = new_interp
+    return state.interp
+end
+
+mode_description(::RecursiveInterpreter) = "interpreted (all code is interpreted, breakpoints always work)"
+mode_description(::MixedInterpreter) = "mixed (code outside the focus modules runs natively unless it can reach them)"
+mode_description(::NonRecursiveInterpreter) = "compiled (stepped-over code runs natively, breakpoints in it are missed)"
+
 toggle_lowered(state) = state.lowered_status = !state.lowered_status
 
 function active_frame(state)

src/commands.jl

@@ -43,13 +43,18 @@ function execute_command(state::DebuggerState, v::Union{Val{:c},Val{:nc},Val{:n}
     assert_allow_step(state) || return false
     cmd == "so" && (cmd = "finish")
     cmd == "u" && (cmd = "until")
+    maybe_warn_native_suppressed(state)
     ret = debug_command(state.interp, state.frame, Symbol(cmd); kwargs...)
     if ret === nothing
         state.overall_result = get_return(root(state.frame))
         state.frame = nothing
         return false
     else
         state.frame, pc = ret
+        # an explicit step-in may land in a module outside the focus set
+        if v isa Union{Val{:s},Val{:si},Val{:sg},Val{:sl}}
+            maybe_grow_focus!(state)
+        end
         if pc isa BreakpointRef
             if pc.stmtidx != 0 # This is the dummy breakpoint to stop just after entering a call
                 if state.terminal !== nothing # fix this, it happens when a test hits this and hasn't set a terminal
@@ -282,6 +287,98 @@ function execute_command(state::DebuggerState, v::Union{Val{:bp}}, cmd::Abstract
 end
 
 
+function execute_command(state::DebuggerState, ::Val{:mode}, cmd::AbstractString)
+    cmds = split(cmd, r" +")
+    io = output_stream(state)
+    if length(cmds) == 1
+        println(io, "Stepping mode: ", mode_description(state.interp))
+        if state.interp isa MixedInterpreter
+            reason = native_suppressed_reason()
+            if reason === nothing
+                println(io, "Focus (interpreted) modules: ",
+                        join(nameof.(interpreted_modules()), ", "), " (manage with `focus`)")
+            else
+                printstyled(io, "Fast path suspended, running fully interpreted ($reason)\n"; color=:yellow)
+            end
+        end
+        return false
+    elseif length(cmds) == 2
+        mode = Symbol(cmds[2])
+        if mode in (:interpreted, :mixed, :compiled)
+            set_session_mode!(state, mode)
+            println(io, "Stepping mode: ", mode_description(state.interp))
+            return false
+        end
+    end
+    return invalid_command(state, cmd)
+end
+
+function _resolve_module(state, name::AbstractString)
+    ex = try
+        Meta.parse(name)
+    catch
+        return nothing
+    end
+    for m in (state.frame === nothing ? (Main,) : (moduleof(active_frame(state)), Main))
+        val = try
+            Core.eval(m, ex)
+        catch
+            continue
+        end
+        val isa Module && return val
+    end
+    # fall back to loaded packages without a binding in the above modules
+    matches = Set{Module}()
+    for (_, m) in Base.loaded_modules
+        string(nameof(m)) == name && push!(matches, m)
+    end
+    length(matches) == 1 && return first(matches)
+    if length(matches) > 1
+        printstyled(stderr, "Multiple loaded modules named $name\n"; color=Base.error_color())
+    end
+    return nothing
+end
+
+function execute_command(state::DebuggerState, ::Val{:focus}, cmd::AbstractString)
+    cmds = split(cmd, r" +")
+    io = output_stream(state)
+    function print_focus()
+        println(io, "Focus set (modules that are interpreted; everything else runs natively when possible):")
+        for (i, m) in enumerate(interpreted_modules())
+            println(io, " [$i] $m")
+        end
+    end
+    if length(cmds) == 1
+        if menus_available(state)
+            focus_menu(state)
+        else
+            print_focus()
+        end
+        return false
+    elseif length(cmds) == 3 && cmds[2] in ("add", "rm")
+        local mod
+        i = cmds[2] == "rm" ? tryparse(Int, cmds[3]) : nothing
+        if i !== nothing
+            mods = interpreted_modules()
+            if !(1 <= i <= length(mods))
+                printstyled(stderr, "No module numbered $i in the focus set\n"; color=Base.error_color())
+                return false
+            end
+            mod = mods[i]
+        else
+            mod = _resolve_module(state, cmds[3])
+            if mod === nothing
+                printstyled(stderr, "Could not resolve $(repr(cmds[3])) as a loaded module\n"; color=Base.error_color())
+                return false
+            end
+        end
+        cmds[2] == "add" ? focus_module!(mod) : unfocus_module!(mod)
+        print_focus()
+        return false
+    end
+    return invalid_command(state, cmd)
+end
+
 function execute_command(state::DebuggerState, _, cmd)
     display(Markdown.parse("""Unknown command `$cmd`. Run `?` to obtain help."""))
     return false
@@ -297,6 +394,10 @@ function execute_command(state::DebuggerState, ::Union{Val{:help}, Val{:?}}, cmd
             - `o`: open the current line in an editor\\
             - `q`: quit the debugger, returning `nothing`\\
             - `C`: toggle compiled mode\\
+            - `M`: toggle mixed mode (code outside the focus modules runs natively unless it can reach them)\\
+            - `mode [interpreted|mixed|compiled]`: show or set the stepping mode\\
+            - `focus`: show the focus set (the modules mixed mode interprets) as an interactive menu\\
+            - `focus add Mod`/`focus rm Mod|i`: add/remove a module (by name or number) from the focus set\\
             - `L`: toggle showing lowered code instead of source code\\
             - `T`: cycle how variable types are shown (compact, none, types only, full)\\
             - `S`: toggle "sticky" (full-screen) mode, on by default: the debugger runs on the terminal's alternate screen and redraws the status in place\\

src/config.jl

@@ -23,7 +23,7 @@ const STICKY = Ref(true)
 
 const CHARSET = Ref{Symbol}(:unicode)
 
-# Interactive TerminalMenus-based menus for `bp`, `f` and `w`
+# Interactive TerminalMenus-based menus for `bp`, `f`, `w` and `focus`
 const INTERACTIVE_MENUS = Ref(true)
 
 set_theme(theme::String) = _current_theme[] = theme
@@ -53,7 +53,7 @@ settings as a `NamedTuple`; keywords set the corresponding option:
   screen (restored on quit) and redraws the status in place instead of scrolling
   (default: `true`). Can be toggled with the `S` key in the debugger.
 - `charset::Symbol`: `:unicode` or `:ascii` (default: `:unicode`)
-- `menus::Bool`: use interactive menus for `bp`, `f` and `w` (default: `true`)
+- `menus::Bool`: use interactive menus for `bp`, `f`, `w` and `focus` (default: `true`)
 """
 function config(; theme::Union{Nothing,String} = nothing,
                   highlight::Union{Nothing,Bool} = nothing,