MhEdZddlZddlZddlmZddlmZddlmZm Z GddZ Gdd ej Z Gd d e Z Gd d e ZdS)a Manage background (threaded) jobs conveniently from an interactive shell. This module provides a BackgroundJobManager class. This is the main class meant for public usage, it implements an object which can create and manage new background jobs. It also provides the actual job classes managed by these BackgroundJobManager objects, see their docstrings below. This system was inspired by discussions with B. Granger and the BackgroundCommand class described in the book Python Scripting for Computational Science, by H. P. Langtangen: http://folk.uio.no/hpl/scripting (although ultimately no code from this text was used, as IPython's system is a separate implementation). An example notebook is provided in our documentation illustrating interactive use of the system. N) get_ipython)AutoFormattedTB)errordebugceZdZdZdZedZedZedZdZ dZ dZ d Z d Z d Zd ZddZdZdZdZdZddZdS)BackgroundJobManageraClass to manage a pool of backgrounded threaded jobs. Below, we assume that 'jobs' is a BackgroundJobManager instance. Usage summary (see the method docstrings for details): jobs.new(...) -> start a new job jobs() or jobs.status() -> print status summary of all jobs jobs[N] -> returns job number N. foo = jobs[N].result -> assign to variable foo the result of job N jobs[N].traceback() -> print the traceback of dead job N jobs.remove(N) -> remove (finished) job N jobs.flush() -> remove all finished jobs As a convenience feature, BackgroundJobManager instances provide the utility result and traceback methods which retrieve the corresponding information from the jobs list: jobs.result(N) <--> jobs[N].result jobs.traceback(N) <--> jobs[N].traceback() While this appears minor, it allows you to use tab completion interactively on the job manager instance. cg|_g|_g|_i|_g|_g|_t j|_t j |_ t j |_ t j |_d|_dS)Nr)_running _completed_deadall _comp_report _dead_reportBackgroundJobBasestat_created_c _s_createdstat_running_c _s_runningstat_completed_c _s_completed stat_dead_c_s_dead_current_job_idselfs Z/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/IPython/lib/backgroundjobs.py__init__zBackgroundJobManager.__init__Isg  -<-<->-9  c8||jSN)_update_statusr rs rrunningzBackgroundJobManager.running[s }rc8||jSr )r!r rs rdeadzBackgroundJobManager.dead`s zrc8||jSr )r!r rs r completedzBackgroundJobManager.completedes rct|r'|di}t|g|Ri|}nt|tr}|s#t jd}|j|j}}nFt|dkr |dx}}n(t|dkr|\}}ntdt|||}ntd|ddrd |_ |j|_|xjdz c_|j|||j|j<t'd |jz||S) a@ Add a new background job and start it in a separate thread. There are two types of jobs which can be created: 1. Jobs based on expressions which can be passed to an eval() call. The expression must be given as a string. For example: job_manager.new('myfunc(x,y,z=1)'[,glob[,loc]]) The given expression is passed to eval(), along with the optional global/local dicts provided. If no dicts are given, they are extracted automatically from the caller's frame. A Python statement is NOT a valid eval() expression. Basically, you can only use as an eval() argument something which can go on the right of an '=' sign and be assigned to a variable. For example,"print 'hello'" is not valid, but '2+3' is. 2. Jobs given a function object, optionally passing additional positional arguments: job_manager.new(myfunc, x, y) The function is called with the given arguments. If you need to pass keyword arguments to your function, you must supply them as a dict named kw: job_manager.new(myfunc, x, y, kw=dict(z=1)) The reason for this asymmetry is that the new() method needs to maintain access to its own keywords, and this prevents name collisions between arguments to new() and arguments to your own functions. In both cases, the result is stored in the job.result field of the background job object. You can set `daemon` attribute of the thread by giving the keyword argument `daemon`. Notes and caveats: 1. All threads running share the same standard output. Thus, if your background jobs generate output, it will come out on top of whatever you are currently writing. For this reason, background jobs are best used with silent functions which simply return their output. 2. Threads also all work within the same global namespace, and this system does not lock interactive variables. So if you send job to the background which operates on a mutable object for a long time, and start modifying that same mutable object interactively (or in another backgrounded job), all sorts of bizarre behaviour will occur. 3. If a background job is spending a lot of time inside a C extension module which does not release the Python Global Interpreter Lock (GIL), this will block the IPython prompt. This is simply because the Python interpreter can only switch between threads at Python bytecodes. While the execution is inside C code, the interpreter must simply wait unless the extension module releases the GIL. 4. There is no way, due to limitations in the Python threads library, to kill a thread once it has started.kwrz4Expression jobs take at most 2 args (globals,locals)zinvalid args for new jobdaemonFTz'Starting job # %s in a separate thread.)callablegetBackgroundJobFunc isinstancestrsys _getframe f_globalsf_localslen ValueErrorBackgroundJobExpr TypeErrorr+rnumr"appendr rstart) r func_or_expargskwargsr(jobframegloblocs rnewzBackgroundJobManager.newjszB K  8**T"%%B#K;;;;;;CC  S ) ) 8 N a((!OU^cTA!!W$ssTASS LNNN#Ks;;CC677 7 ::h & & CJ& ! C    7#'ABBB  rcXt|tr|n|j}|j|Sr )r/intr9r )rjob_keyr9s r __getitem__z BackgroundJobManager.__getitem__s(#GS11Bggw{x}rc*|S)zAn alias to self.status(), This allows you to simply call a job manager instance much like the Unix `jobs` shell command.)statusrs r__call__zBackgroundJobManager.__call__s {{}}rc|j|j|j}}}|j|j|j}}}t |D]\}}|j} | |kr| |kr5|||j |d||<N| |kr4|||j |d||<td||dd<dS)aUpdate the status of the job lists. This method moves finished jobs to one of two lists: - self.completed: jobs which completed successfully - self.dead: jobs which finished but died. It also copies those jobs to corresponding _report lists. These lists are used to report jobs completed/dead since the last update, and are then cleared by the reporting function after each call.FN) rrrr r r enumerate stat_coder:rrfilter) rsrunscompsdeadr"r&r$r9r?stats rr!z#BackgroundJobManager._update_statuss"_d.?Ue$(=$/4:D"'** % %HC=Dt||  %%%!((---$  C   !((---$ D'** rc|r@td|z|D]}t|jd|tdSdS)zYReport summary for a given job group. Return True if the group had any elements.z%s jobs:z : TN)printr9)rgroupnamer?s r _group_reportz"BackgroundJobManager._group_reportsd   *t# $ $ $ 1 1377733/0000 GGG4   rc t|}|r:ddi|d}td|d|d|dg|dd<d SdS) zKFlush a given job group Return True if the group had any elements.r)sz Flushing  z job.NT)r5 setdefaultrT)rrUrVnjobsplurals r _group_flushz!BackgroundJobManager._group_flushso E   V&&uS11F EUUU444? @ @ @E!!!H4   rc|||jd}||jd}g|jdd<g|jdd<|p|S)zPrint the status of newly finished jobs. Return True if any new jobs are reported. This call resets its own state every time, so it only reports jobs which have finished since the last time it was called. Completedz'Dead, call jobs.traceback() for detailsN)r!rWrr)rnew_compnew_deads r _status_newz BackgroundJobManager._status_new sz %%d&7EE%%d&7&OQQ!!!!!!!!#8#rrc|||jd||jd||jdg|jdd<g|jdd<dS)z3Print a status of all jobs currently being managed.RunningrbDeadN)r!rWr"r&r$rr)rverboses rrIzBackgroundJobManager.statuss  4< 222 4>+666 49V,,,!!!!!!!!rcP |j|}|j}||jkrtd|zdS||jkr|j|dS||jkr|j|dSdS#t$rtd|zYdSwxYw)z*Remove a finished (completed or dead) job.z0Job #%s is still running, it can not be removed.NJob #%s not found) r rMrrrr&removerr$KeyError)rr9r?rMs rrlzBackgroundJobManager.remove's &(3-C IDO++H3NOOOd///%%c*****dl**   %%%%%+* - - - %+ , , , , , , -s BB%$B%c|j}|j|jzD] }||j= ||jd}||jd}|s|st ddSdSdS)aFlush all finished jobs (completed and dead) from lists. Running jobs are never flushed. It first calls _status_new(), to update info. If any jobs have completed since the last _status_new() call, the flush operation aborts.rbrhzNo jobs to flush.N)r r&r$r9r`rT)ralljobsr?fl_compfl_deads rflushzBackgroundJobManager.flush8s(>$)+ " "C  ##DNK@@##DIv66 '7 ' % & & & & & ' ' ' 'rcn |j|jS#t$rtd|zYdSwxYw)z(result(N) -> return the result of job N.rkN)r resultrmr)rr9s rrtzBackgroundJobManager.resultLsN -8C=' ' - - - %+ , , , , , , -s 44ct|tr|n|j} |j|dS#t $rt d|zYdSwxYw)Nrk)r/rEr9r tracebackrmr)rr?r9s r _tracebackzBackgroundJobManager._tracebackSswS))6ccsw - HSM # # % % % % % - - - %+ , , , , , , -sAA! A!Nc|U||jD]7}td|z||t8dS||dS)NzTraceback for: %r)r!r$rTrw)rr?deadjobs rrvzBackgroundJobManager.tracebackZs ;    ! ! !9  )G3444(((   OOC r)rr )__name__ __module__ __qualname____doc__rpropertyr"r$r&rCrGrJr!rWr`rerIrlrrrtrwrvrrrr)sP>!!!$XXX[[[z+++B      $$$ " " " "&&&"'''(------!!!!!!rrcVeZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z d Zd ZdZdZdS)raBase class to build BackgroundJob classes. The derived classes must implement: - Their own __init__, since the one here raises NotImplementedError. The derived constructor must call self._init() at the end, to provide common initialization. - A strform attribute used in calls to __str__. - A call() method, which will make the actual execution call and must return a value to be held in the 'result' field of the job object. Createdrrgr)rbr*z3Dead (Exception), call jobs.traceback() for detailsc td)ztMust be implemented in subclasses. Subclasses must call :meth:`_init` for standard initialisation. z,This class can not be instantiated directly.)NotImplementedErrorrs rrzBackgroundJobBase.__init__}s ""PQQQrcdD]}t||s Jd|z d|_tj|_tj|_d|_d|_ tj j n#tddd j YnxYwfd |_ d|_tj|dS) z3Common initialization for all BackgroundJob objects)callstrformzMissing attribute <%s>NFz!Contextnocolorr))mode color_scheme tb_offsetcdddSr r)make_tbsrz)BackgroundJobBase._init..stT!:!:r)hasattrr9r stat_createdrIrrMfinishedrtr InteractiveTBtextr_make_tb_tb threadingThreadr)rattrrs @r_initzBackgroundJobBase._inits' G GD4%% F F'?$'F F F F F*7 *9 <  !mm16GG %Y! GG ;::: !!$'''''s A55Bc|jSr )rrs r__str__zBackgroundJobBase.__str__s |rc$d|j|jfzS)Nz)r9rrs r__repr__zBackgroundJobBase.__repr__s(DHdl+CCCrc.t|jdSr )rTrrs rrvzBackgroundJobBase.tracebacks dhrcr tj|_tj|_||_tj|_tj|_d|_ dS#tj |_tj |_d|_ d|_| |_ YdSxYw)NTz7)r stat_runningrIrrMrrtstat_completedrr stat_deadrrrrs rrunzBackgroundJobBase.runs ".;DK.=DN!YY[[DK/=DK.?DN!DMMM -.8DK.:DN!DMWDK!]]__DHHHHs ;A((A B6N)rzr{r|r}rrrrrrrrrrrrrvrrrrrres  "LL NQ"2JIKRRR(((>DDD"""""rrc eZdZdZddZdZdS)r7zDEvaluate an expression as a background job (uses a separate thread).Nct|dd|_|in|}|in|}|x|_|_||_||_|dS)zCreate a new job from a string which can be fed to eval(). global/locals dicts can be provided, which will be passed to the eval call.zevalN)compilecode expressionrrArBr)rrrArBs rrzBackgroundJobExpr.__init__s_J'DVLL \rrtKbbS)33$,  rcBt|j|j|jSr )rrrArBrs rrzBackgroundJobExpr.callsDIdi111r)NNrzr{r|r}rrrrrr7r7s=NN 22222rr7ceZdZdZdZdZdS)r.zARun a function call as a background job (uses a separate thread).ct|std||_||_||_t ||_|dS)zCreate a new job from a callable object. Any positional arguments and keyword args given to this constructor after the initial callable are passed directly to it.z4first argument to BackgroundJobFunc must be callableN)r,r8funcr=r>r0rr)rrr=r>s rrzBackgroundJobFunc.__init__sb ~~ HFHH H   4yy  rc0|j|ji|jSr )rr=r>rs rrzBackgroundJobFunc.callsty$)3t{333rNrrrrr.r.s8KK&44444rr.)r}r1rIPythonrIPython.core.ultratbrloggingrrrrrr7r.rrrrs)> 000000 y!y!y!y!y!y!y!y!x U"U"U"U"U" (U"U"U"p22222)222.44444)44444r