
    X-Ph                    6   U d Z ddlmZ ddlZddlZddlZddlZddlmZ ddl	m
Z
mZmZmZ ddlmZ ddlmZmZmZmZmZmZmZmZ ddlmZ dd	lmZ dd
lmZ ddl m!Z! ddl"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)m*Z* ddl+m,Z, ddl-m.Z.m/Z/ ddl0m1Z1m2Z2m3Z3 ddl4m5Z5 ddl6m7Z7m8Z8 ddl9m:Z:m;Z; ddl<m=Z= ddl>m?Z?m@Z@ ddlAmBZB ddlCmDZDmEZEmFZF dZGdeHd<   dZI G d d          ZJd|d%ZKd}d+ZLd~d,ZM G d- d.e          ZN G d/ d0e          ZOeeNeOf         ZPd1eHd2<   dd:ZQdd>ZRdd@ZSddAZTddBZUddGZVddMZWddOZXddTZYddWZZddZZ[dd_Z\dddZ]ddgZ^ddhZ_ddlZ`eja        dmk    rdnZbdeHdo<   n+ejc        dpz   ejc        dqz   ejd        dpz   ejd        dqz   fZbdeHdo<   dduZeddwZfddxZgdd{ZhdS )aR  Update build by processing changes using fine-grained dependencies.

Use fine-grained dependencies to update targets in other modules that
may be affected by externally-visible changes in the changed modules.

This forms the core of the fine-grained incremental daemon mode. This
module is not used at all by the 'classic' (non-daemon) incremental
mode.

Here is some motivation for this mode:

* By keeping program state in memory between incremental runs, we
  only have to process changed modules, not their dependencies. The
  classic incremental mode has to deserialize the symbol tables of
  all dependencies of changed modules, which can be slow for large
  programs.

* Fine-grained dependencies allow processing only the relevant parts
  of modules indirectly affected by a change. Say, if only one function
  in a large module is affected by a change in another module, only this
  function is processed. The classic incremental mode always processes
  an entire file as a unit, which is typically much slower.

* It's possible to independently process individual modules within an
  import cycle (SCC). Small incremental changes can be fast independent
  of the size of the related SCC. In classic incremental mode, any change
  within a SCC requires the entire SCC to be processed, which can slow
  things down considerably.

Some terms:

* A *target* is a function/method definition or the top level of a module.
  We refer to targets using their fully qualified name (e.g.
  'mod.Cls.method'). Targets are the smallest units of processing during
  fine-grained incremental checking.

* A *trigger* represents the properties of a part of a program, and it
  gets triggered/fired when these properties change. For example,
  '<mod.func>' refers to a module-level function. It gets triggered if
  the signature of the function changes, or if the function is removed,
  for example.

Some program state is maintained across multiple build increments in
memory:

* The full ASTs of all modules are stored in memory all the time (this
  includes the type map).

* A fine-grained dependency map is maintained, which maps triggers to
  affected program locations (these can be targets, triggers, or
  classes). The latter determine what other parts of a program need to
  be processed again due to a fired trigger.

Here's a summary of how a fine-grained incremental program update happens:

* Determine which modules have changes in their source code since the
  previous update.

* Process changed modules one at a time. Perform a separate full update
  for each changed module, but only report the errors after all modules
  have been processed, since the intermediate states can generate bogus
  errors due to only seeing a partial set of changes.

* Each changed module is processed in full. We parse the module, and
  run semantic analysis to create a new AST and symbol table for the
  module. Reuse the existing ASTs and symbol tables of modules that
  have no changes in their source code. At the end of this stage, we have
  two ASTs and symbol tables for the changed module (the old and the new
  versions). The latter AST has not yet been type checked.

* Take a snapshot of the old symbol table. This is used later to determine
  which properties of the module have changed and which triggers to fire.

* Merge the old AST with the new AST, preserving the identities of
  externally visible AST nodes for which we can find a corresponding node
  in the new AST. (Look at mypy.server.astmerge for the details.) This
  way all external references to AST nodes in the changed module will
  continue to point to the right nodes (assuming they still have a valid
  target).

* Type check the new module.

* Take another snapshot of the symbol table of the changed module.
  Look at the differences between the old and new snapshots to determine
  which parts of the changed modules have changed. The result is a set of
  fired triggers.

* Using the dependency map and the fired triggers, decide which other
  targets have become stale and need to be reprocessed.

* Create new fine-grained dependencies for the changed module. We don't
  garbage collect old dependencies, since extra dependencies are relatively
  harmless (they take some memory and can theoretically slow things down
  a bit by causing redundant work). This is implemented in
  mypy.server.deps.

* Strip the stale AST nodes that we found above. This returns them to a
  state resembling the end of semantic analysis pass 1. We'll run semantic
  analysis again on the existing AST nodes, and since semantic analysis
  is not idempotent, we need to revert some changes made during semantic
  analysis. This is implemented in mypy.server.aststrip.

* Run semantic analyzer passes 2 and 3 on the stale AST nodes, and type
  check them. We also need to do the symbol table snapshot comparison
  dance to find any changes, and we need to merge ASTs to preserve AST node
  identities.

* If some triggers haven been fired, continue processing and repeat the
  previous steps until no triggers are fired.

This is module is tested using end-to-end fine-grained incremental mode
test cases (test-data/unit/fine-grained*.test).
    )annotationsN)Sequence)CallableFinal
NamedTupleUnion)	TypeAlias)DEBUG_FINE_GRAINEDFAKE_ROOT_MODULEBuildManagerBuildResultGraphState
load_graphprocess_fresh_modules)FineGrainedDeferredNode)CompileError)FileSystemCache)BuildSource)	DecoratorFuncDef
ImportFromMypyFileOverloadedFuncDef
SymbolNodeSymbolTableTypeInfo)Options)semantic_analysis_for_sccsemantic_analysis_for_targets)SymbolSnapshotcompare_symbol_table_snapshotssnapshot_symbol_table)
merge_asts)SavedAttributesstrip_target)get_dependencies_of_targetmerge_dependencies)trigger_to_target)WILDCARD_TAGmake_trigger)
type_state)is_stdlib_filemodule_prefixsplit_targeti  r   MAX_ITER)mypy_extensionstyping_extensionsc                  B    e Zd ZddZ	 dd dZd!dZd"dZd#dZd$dZdS )%FineGrainedBuildManagerresultr   returnNonec                   |j         }|| _         |j        | _        t          | j                  | _        |j        | _        t          |                    t                    | j                   |j	        
                                | _        |j	                                        | _        d| _        g | _        d| j         _        g | _        g | _        g | _        g | _        dS )aY  Initialize fine-grained build based on a batch build.

        Args:
            result: Result from the initialized build.
                    The manager and graph will be taken over by this class.
            manager: State of the build (mutated by this class)
            graph: Additional state of the build (mutated by this class)
        NF)managergraphget_module_to_path_mapprevious_modulesfg_depsdepsr(   load_fine_grained_depsr   errorstargetsprevious_targets_with_errorscopyprevious_messagesblocking_errorstalecache_enabled	triggeredchanged_modulesupdated_modulesprocessed_targets)selfr5   r9   s      R/var/www/html/test/jupyter/venv/lib/python3.11/site-packages/mypy/server/update.py__init__z FineGrainedBuildManager.__init__   s     .\
 6tz B BO	799:JKKTYWWW,3N,B,B,D,D),2M,>,>,@,@6:,.
 &+" %'68*,,.    FrI   list[tuple[str, str]]removed_modulesfollowedbool	list[str]c           	        | j                                          ||z   }d |D             }|| _        |s| j        S | j        j                                         g | _        g | _        t          || j	        z             }d |D             }| j        
                    dd                    d |D                       z             | j        rCt          | j                  r/| j        
                    dt          | j                  z             d}| j        rT| j        
                    d| j        d	                     t          | j        g|z             }| j        d	         }d| _        	 |                     |||||          }|\  }\  }}	}
|
||	f| _        || _	        |
}n|st#          | j        | j        | j        t)                      |h| j        | j                   }t          |          }|sB| j        j                                        | _        | j        j                                        }nt1          || j                  }|                                | _        |S )
aJ  Update previous build result by processing changed modules.

        Also propagate changes to other modules as needed, but only process
        those parts of other modules that are affected by the changes. Retain
        the existing ASTs and symbol tables of unaffected modules.

        Reuses original BuildManager and Graph.

        Args:
            changed_modules: Modules changed since the previous update/build; each is
                a (module id, path) tuple. Includes modified and added modules.
                Assume this is correct; it's not validated here.
            removed_modules: Modules that have been deleted since the previous update
                or removed from the build.
            followed: If True, the modules were found through following imports

        Returns:
            A list of errors.
        c                    h | ]\  }}|S  rW   .0module_s      rM   	<setcomp>z1FineGrainedBuildManager.update.<locals>.<setcomp>   s    ???)&!v???rO   c                    h | ]\  }}|S rW   rW   rY   idr[   s      rM   r\   z1FineGrainedBuildManager.update.<locals>.<setcomp>   s    777eb!r777rO   z==== update %s ====z, c              3  :   K   | ]\  }}t          |          V  d S N)reprr^   s      rM   	<genexpr>z1FineGrainedBuildManager.update.<locals>.<genexpr>   s,      -T-T52qd2hh-T-T-T-T-T-TrO   z previous targets with errors: %sNzexisting blocker: r   )rK   clearrI   rD   r9   find_module_cacherH   rJ   dedupe_modulesrF   log_fine_grainedjoinrB   
is_verbosesortedrE   
update_one$propagate_changes_using_dependenciesr:   r>   setr@   rA   new_messages#sort_messages_preserving_file_orderrC   )rL   rI   rQ   rR   removed_setinitial_setrE   r5   next_id	next_pathblocker_messagesmessagess               rM   updatezFineGrainedBuildManager.update   s   2 	$$&&&)O;?????. 	*)) 	&,,...!(4:)EFF77777%%!DII-T-TO-T-T-T$T$TT	
 	
 	
 , 	DL1I1I 	L))2VD<]5^5^^    	' L))*Wt?RST?U*W*WXXX,d.A-B_-TUUO!03N"&D$	__k>8 F GMCO1gy3C+'.	&:#,
+ # 
 #GLJIEEI5*# # #1"A"A& 8<8K8S8S8U8UD5#|2??AAHI$	L 7xAWXX!)rO   targetstrc           	        | j         j                                         t          | j         | j        | j        t                      t                      | j        |hz  g           }| j         j                                        | _        | j         j        	                                
                                | _        |                     |g           S )zkTrigger a specific target explicitly.

        This is intended for use by the suggestions engine.
        )r9   r@   resetrl   r:   r>   rm   rB   rA   rn   rC   rD   rv   )rL   rw   rI   s      rM   triggerzFineGrainedBuildManager.trigger2  s    
 	!!###>LJIEEEE-8
 
 -1L,?,G,G,I,I)!%!4!A!A!C!C!H!H!J!J{{?B///rO   c                B    | j         j                                         dS )zFlush AST cache.

        This needs to be called after each increment, or file changes won't
        be detected reliably.
        N)r9   	ast_cacherd   )rL   s    rM   flush_cachez#FineGrainedBuildManager.flush_cacheF  s!     	$$&&&&&rO   rq   set[str]rp   rE   
str | None?tuple[list[tuple[str, str]], tuple[str, str], list[str] | None]c           
        t          j                     }|                    d          \  }|k    r2| j        vr)|vr%| j                            dd           ||fdfS |                     ||v |          }|\  }	\  }}
fd|D             }t          |	|z             }t          j                     }| j                            d d||z
  dd	t          |           d
           ||f|
fS )aR  Process a module from the list of changed modules.

        Returns:
            Tuple with these items:

            - Updated list of pending changed modules as (module id, path) tuples
            - Module which was actually processed as (id, path) tuple
            - If there was a blocking error, the error messages from it
        r   zskip z1 (module with blocking error not in import graph)Nc                *    g | ]\  }}|k    ||fS rW   rW   )rY   r_   pathrr   s      rM   
<listcomp>z6FineGrainedBuildManager.update_one.<locals>.<listcomp>q  s&    WWW("dwB:rO   zupdate once: z in z.3fzs - z left)timepopr<   r9   rg   update_modulerf   len)rL   rI   rq   rp   rE   rR   t0rs   r5   	remainingrt   t1rr   s               @rM   rk   z"FineGrainedBuildManager.update_oneN  sM   " Y[[,0033 ~%%t444{**L))TTTT   #Wi$8$>>##GY;8NPXYY<B9	'GY)9WWWWoWWW(_)DEEY[[%%UGUUbUUU#o:N:NUUU	
 	
 	
 ) 46FFFrO   rZ   r   force_removedc           
        | j                             d|d           | j                            |           t	          | j         j        j        |          s	|t          v rg ||fdfS | j         }| j        }| j	        }t          || j        |           t          |||g           t          j                    }i }	||j        v r%t          ||j        |         j                  }
|
|	|<   |j                                         | j                            |           t)          |||||||          }t+          |t,                    r"|\  }}}}t/          |          | _        |||f|fS t+          |t0                    sJ |\  }}}}t          j                    }t3          ||	||i          }t5          | j                   r6d |D             }| j                             dt7          |                     | j                            || j        z             ||v r:||                             | j                   ||                                           |tC          ||| j        ||htE                      | j                  z  }t          j                    }|#                    ||z
  ||z
             | j        $                    |j        %                                           t/          |          | _        |||fdfS )a  Update a single modified module.

        If the module contains imports of previously unseen modules, only process one of
        the new modules and return the remaining work to be done.

        Args:
            module: Id of the module
            path: File system path of the module
            force_removed: If True, consider module removed from the build even if path
                exists (used for removing an existing file from the build)
            followed: Was this found via import following?

        Returns:
            Tuple with these items:

            - Remaining modules to process as (module id, path) tuples
            - Module which was actually processed as (id, path) tuple
            - If there was a blocking error, the error messages from it
        z--- update single z ---Nc                <    g | ]}|                     d           |S )z__>)endswith)rY   r{   s     rM   r   z9FineGrainedBuildManager.update_module.<locals>.<listcomp>  s,    XXXG@P@PQV@W@WXXXXrO   triggered: )targets_with_errorsrK   )update_isolated_timepropagate_time)&r9   rg   rJ   appendr-   optionsabs_custom_typeshed_dirSENSITIVE_INTERNAL_MODULESr<   r:   ensure_deps_loadedr>   ensure_trees_loadedr   modulesr#   namesr@   rz   rK   update_module_isolated
isinstanceBlockedUpdater;   NormalUpdatecalculate_active_triggersri   rj   rH   extendrB   update_fine_grained_deps
free_staterl   rm   	add_statsrv   rA   )rL   rZ   r   r   rR   r9   r<   r:   r   old_snapshotssnapshotr5   r   r@   treer   rH   filteredt2s                      rM   r   z%FineGrainedBuildManager.update_module{  s    , 	%%&I6&I&I&IJJJ##F+++ 4</GNN	,333~t++,0
649e444 	GUVH555Y[[>@W_$$,VW_V5L5RSSH$,M&!%%f---'D'#3UM8
 
 fm,, 	5.4+FD)V$:5$A$AD!vtnf44&,/////(.%i Y[[-g}vtnUU	dl## 	NXXyXXXHL))*Lx8H8H*L*LMMMi$*KKLLLU??&M2249===&M$$&&&9IH #"4
 
 
 	
	 Y[[rBwrBwOOO 	)001G1G1I1IJJJ 6u = =64.$..rO   N)r5   r   r6   r7   )F)rI   rP   rQ   rP   rR   rS   r6   rT   )rw   rx   r6   rT   r6   r7   )rI   rP   rq   r   rp   r   rE   r   rR   rS   r6   r   )
rZ   rx   r   rx   r   rS   rR   rS   r6   r   )	__name__
__module____qualname__rN   rv   r{   r~   rk   r   rW   rO   rM   r4   r4      s        "/ "/ "/ "/P 	a a a a aF0 0 0 0(' ' ' '+G +G +G +GZZ/ Z/ Z/ Z/ Z/ Z/rO   r4   r9   r   r:   dict[str, State]initialSequence[str]r6   rT   c                P   t          |          }t                      }g }|r|                                }||v s||vr|                    |           || j        vrG||         j        pg }|                    ||         j        |z              |                    |           ||S )a  Find all the deps of the nodes in initial that haven't had their tree loaded.

    The key invariant here is that if a module is loaded, so are all
    of their dependencies. This means that when we encounter a loaded
    module, we don't need to explore its dependencies.  (This
    invariant is slightly violated when dependencies are added, which
    can be handled by calling find_unloaded_deps directly on the new
    dependencies.)
    )	listrm   r   addr   	ancestorsr   dependenciesr   )r9   r:   r   worklistseenunloadednoder   s           rM   find_unloaded_depsr     s     G}}HUUDH
 "||~~4<<4u,,w&&d-3IOOE$K4y@AAAOOD!!!  " OrO   rZ   rx   r>   dict[str, set[str]]r7   c                `   | |v r||          j         rdS |                     d          }t          t          |                    D ]h}d                    |d|dz                      }||v rB||         j         s5t          ||                                         |           d||         _         idS )a  Ensure that the dependencies on a module are loaded.

    Dependencies are loaded into the 'deps' dictionary.

    This also requires loading dependencies from any parent modules,
    since dependencies will get stored with parent modules when a module
    doesn't exist.
    N.   T)fine_grained_deps_loadedsplitranger   rh   r(   r?   )rZ   r>   r:   partsibases         rM   r   r     s     5=ALLE3u:: 8 8xxgAg''5==t!E=uT{AACCTJJJ37E$K0	8 8rO   c                    t          | ||          }|ret          |           rC|                     d                    t	          |          t          |                               t          |||            dS dS )zDEnsure that the modules in initial and their deps have loaded trees.z4Calling process_fresh_modules on set of size {} ({})N)r   ri   rg   formatr   rj   r   )r9   r:   r   
to_processs       rM   r   r     s     $GUG<<J :g 	$$FMM
OOVJ%7%7   
 	eZ99999: :rO   c                  8    e Zd ZU ded<   ded<   ded<   ded<   dS )	r   rx   rZ   r   rP   r   zMypyFile | Noner   Nr   r   r   __annotations__rW   rO   rM   r   r     s<         KKKIII$$$$rO   r   c                  8    e Zd ZU ded<   ded<   ded<   ded<   dS )	r   rx   rZ   r   rP   r   rT   ru   Nr   rW   rO   rM   r   r   '  s<         KKKIII$$$$rO   r   
_TypeAliasUpdateResultr   r<   dict[str, str]r   r   rS   rR   c                
   | vr                     d|            j                            |          r|r$t          | |           t	          | |g d          S t          j        || |fg|          }| j        v rj                            |            |                     |           j	                            |           dfd}g }		 | v r| = t          ||	           nU# t          $ rH}
|
j        sJ  || gd |	D             z              t          |
j        |g |
j                  cY d}
~
S d}
~
ww xY wt          |          j                   d	 |	D             }t#          |          d
k    r[t%          |          \  } }|                    | |f           |} |d |D                                             d| d           ng }|          }|                                 |j        
J d            t+          j                    }	 t-          |j        gj                   n:# t          $ r-}
 || g           t          | |||
j                  cY d}
~
S d}
~
ww xY w| |j        i}t3          i|           t+          j                    }|                                                                 |                                 |                                 |                                 t+          j                    }|                                 t+          j                    }                     ||z
  ||z
  ||z
             || <   t	          | |||j                  S )a  Build a new version of one changed module only.

    Don't propagate changes to elsewhere in the program. Raise CompileError on
    encountering a blocking error.

    Args:
        module: Changed module (modified, created or deleted)
        path: Path of the changed module
        manager: Build manager
        graph: Build graph
        force_removed: If True, consider the module removed from the build even it the
            file exists

    Returns a named tuple describing the result (see above for details).
    znew module NidsrT   r6   r7   c                    | D ];}|k    rrj         |<   n|j         v rj         |= |k    rr|<   4|v r|= <d S ra   )r   )r   r_   r:   r9   orig_module
orig_state	orig_trees     rM   restorez'update_module_isolated.<locals>.restoreY  s      	 	B[  Y &/##w&&OB'[  Z &b		u"I	 	rO   c                    g | ]	}|j         
S rW   )r_   rY   sts     rM   r   z*update_module_isolated.<locals>.<listcomp>o  s    888bBE888rO   c                *    g | ]}|j         |j        fS rW   )r_   xpathr   s     rM   r   z*update_module_isolated.<locals>.<listcomp>y  s!    ???Rrx(???rO   r   c                    g | ]\  }}|S rW   rW   r^   s      rM   r   z*update_module_isolated.<locals>.<listcomp>  s    333A333rO   z--> z (newly imported)zfile must be at least parsed)semanal_timetypecheck_timefinish_passes_time)r   rT   r6   r7   )!rg   fscacheisfiledelete_moduler   get_sourcesmissing_modulesremovegetr   r   r   module_with_blockerr   ru   r   r   r   find_relative_leaf_module
parse_filer   r   r   r_   r@   !replace_modules_with_new_variantstype_checkerrz   type_check_first_passtype_check_second_passdetect_possibly_undefined_varsfinish_passesr   )rZ   r   r9   r<   r:   r   rR   sourcesr   new_moduleserrrI   remaining_modulesstater   new_modules_dictr   r   t3r   r   r   s     ` `              @@@rM   r   r   1  s@   0 U  !9v!9!9:::?!!$'' 4= 4fdE7333FD"d333'/+;vtn=MxXXG(((&&v...K6""J##F++I           "KNU??f7GUK8888 N N N&&&&88K8888999S4dBMMMMMMMM	N f(BCCC @?;???O ?a0%HH~...+33!2333444  !C!C!C!CDDDD&ME 
:!!#A!!!	BL!%%(W^DDDD L L LVT+<clKKKKKKKKL 5;EJ3G%gu{I6NP`aaa	B	   	!!!	  """	((***	B		B2727WY\^W^___E&M&7DDDs<   C7 7
E	=E>E	E	%I 
I9"I4.I94I9r   rP   tuple[str, str]c                    | sJ t          |           } d | D             }| D ];\  }}||         }t          t          |j                  |z            dk    r||fc S <| d         S )aT  Find a module in a list that directly imports no other module in the list.

    If no such module exists, return the lexicographically first module from the list.
    Always return one of the items in the modules list.

    NOTE: If both 'abc' and 'typing' have changed, an effect of the above rule is that
        we prefer 'abc', even if both are in the same SCC. This works around a false
        positive in 'typing', at least in tests.

    Args:
        modules: List of (module, path) tuples (non-empty)
        graph: Program import graph that contains all modules in the module list
    c                    h | ]\  }}|S rW   rW   rX   s      rM   r\   z,find_relative_leaf_module.<locals>.<setcomp>  s    222YVQ&222rO   r   )rj   r   rm   r   )r   r:   
module_setrZ   r   r   s         rM   r   r     s     NNNWooG22'222J    fs5%&&344994< : 1:rO   	module_idc                   |                     d|            | |v r|| = | |j        v r|j        | = |                     d          }t          |          dk    rPd                    |d d                   }||j        v r*|j        |         }|d         |j        v r|j        |d         = |j                            |          r|j        	                    |            d S d S )Nzdelete module r   r   )
rg   r   r   r   rh   r   r   r   r   r   )r   r   r:   r9   
components	parent_idparents          rM   r   r     s    ;i;;<<<E)GO##OI&%%J
:HHZ_--	''_Y/F"~--LB0 d## /##I...../ /rO   c                    t                      }g }| D ]5\  }}||vr,|                    |           |                    ||f           6|S ra   )rm   r   r   )r   r   r5   r_   r   s        rM   rf   rf     sW    UUDF & &DT>>HHRLLLMM2t*%%%MrO   c                >    d |                                  D             S )Nc                $    i | ]\  }}||j         S rW   r   )rY   rZ   r   s      rM   
<dictcomp>z*get_module_to_path_map.<locals>.<dictcomp>  s     AAA<64FDJAAArO   )items)r:   s    rM   r;   r;     s    AA5;;==AAAArO   r   r   rI   list[BuildSource]c           	         g }|D ]@\  }}|                      |          r&|                    t          ||d |                     A|S )N)rR   )r   r   r   )r   r   rI   rR   r   r_   r   s          rM   r   r     s\     G# K KD>>$ 	KNN;tRIIIJJJNrO   r   $dict[str, dict[str, SymbolSnapshot]]r   dict[str, MypyFile | None]r   c                   t                      }|D ]m}|                    |          }||                    |           i }||         }|2t          |t	                                }|                    |           nt          ||j                  }t          |||          }|                    d          }	|                                D ]}
|
                    d          |	dz   k    r:|
	                    d          d         dvr|                    |t          z              |
                    d          |	dz   k    r7|                    |
                    dd          d         t          z              ||z  }od |D             S )zDetermine activated triggers by comparing old and new symbol tables.

    For example, if only the signature of function m.f is different in the new
    symbol table, return {'<m.f>'}.
    Nr   r   r   )__builtins____file__r   __package____doc__r   c                ,    h | ]}t          |          S rW   r+   rY   names     rM   r\   z,calculate_active_triggers.<locals>.<setcomp>"  s     1114L111rO   )rm   r   r   r#   r   r   r"   countrC   r   r*   rsplit)r9   r   r   r   r_   	snapshot1new	snapshot2diffpackage_nesting_levelitems              rM   r   r     s    eeE ! !!%%b))	IIbMMMI"o;-b+--@@IIIbMMMM-b#)<<I-b)YGG "IIKK 	@ 	@Dzz#"7!";;;

3PR@S \ A A l*+++zz#!6!:::S!,,Q/,>???1151111rO   old_modulesc                    |D ]V}|                     |          }||         }|r5|3t          ||j        ||j                   || j        |<   |||         _        WdS )a  Replace modules with newly builds versions.

    Retain the identities of externally visible AST nodes in the
    old ASTs so that references to the affected modules from other
    modules will still be valid (unless something was deleted or
    replaced with an incompatible definition, in which case there
    will be dangling references that will be handled by
    propagate_changes_using_dependencies).
    N)r   r$   r   r   r   )r9   r:   r  r   r_   preserved_module
new_modules          rM   r   r   %  sy      . .&??2.. _
 	.
 6')9)?ZM]^^^"2GOB-E"IN. .rO   rH   up_to_date_modulesr   rK   c           
     f   d}g }|s|r|dz  }|t           k    rt          dt           z            t          | |||          \  }	}
}|                    fdt	          |
          D                        |D ]s}t          |          }|_||vr[||	vrt                      |	|<   |                     d|            t          | |          \  }}|	|         	                    |           tt                      }|D ]}t          j        |           t	          |	                                d           D ]"\  }}||vsJ |t          | ||||          z  }#t                      }t                      }t          |           r%|                     d	t          |                     |||S )
ai  Transitively rechecks targets based on triggers and the dependency map.

    Returns a list (module id, path) tuples representing modules that contain
    a target that needs to be reprocessed but that has not been parsed yet.

    Processed targets should be appended to processed_targets (used in tests only,
    to test the order of processing targets).
    r   r   z5Max number of iterations (%d) reached (endless loop?)c              3  6   K   | ]}||         j         fV  d S ra   r  )rY   r_   r:   s     rM   rc   z7propagate_changes_using_dependencies.<locals>.<genexpr>]  s.       R R2"eBio!6 R R R R R RrO   Nzprocess target with error: c                    | d         S Nr   rW   )xs    rM   <lambda>z6propagate_changes_using_dependencies.<locals>.<lambda>p  s
    AaD rO   keyr   )r0   RuntimeErrorfind_targets_recursiver   rj   r.   rm   rg   lookup_targetrv   r,   reset_subtype_caches_forr  reprocess_nodesri   r   )r9   r:   r>   rH   r   r   rK   num_iterr   todor   stale_protosrw   r_   
more_nodesr[   infonodess    `                rM   rl   rl   =  s$   $ H/1  %H* %HAhVYaabbb'=UIt-?(
 (
$h 	   R R R RAQAQ R R RRRR * 	, 	,Fuf--B~",>">">T>>"uuDH(()Ov)O)OPPP -gv > >
AR
+++EE	 ! 	6 	6D/5555  

..AAA 	] 	]IB/////%UDJ[\\\II !UU!eeg 	H$$%F4	??%F%FGGGK  %H* %HN rO   triggersGtuple[dict[str, set[FineGrainedDeferredNode]], set[str], set[TypeInfo]]c                   i }|}t                      }t                      }t                      }	|rH||z  }|}
t                      }|
D ],}|                    d          rYt          |t          |                    }|rt	          |||           ||                    |t                                |z
  z  }qt          ||          }|||v r|| j        vs| j        |         j        r|	                    |           ||vrt                      ||<   | 	                    d|            t          | |          \  }}|r|                    |           ||                             |           .|H||	|fS )zFind names of all targets that need to reprocessed, given some triggers.

    Returns: A tuple containing a:
     * Dictionary from module id to a set of stale targets.
     * A set of module ids for unparsed modules with stale targets.
    <Nz	process: )rm   
startswithr.   r)   r   r   r   is_cache_skeletonr   rg   r+  rv   )r9   r:   r4  r>   r   r5   r   	processedr0  unloaded_filescurrentrw   r   deferredstale_protos                  rM   r*  r*  ~  s    79FH%%I"%%%L"uuN
  #3X	55 	3 	3F  %% 3)%1B61J1JKK	 ?&y$>>>DHHVSUU33i??)%88	$ 222W_44y1C 5 #&&y111F**(+F9%(()=V)=)=>>>(5gv(F(F%+ 2 $$[111y!((2222G  #3J ><//rO   nodesetset[FineGrainedDeferredNode]c                   ||vr&|                      d|z             t                      S | j        |         }t          |j        |j                  }d |                                D             }t          |j        |j                  }dd}	t          ||	          }
||         }|j	        }| j
                            |j        |j        |j        p|j                   | j
                            |j        |j                   t                      }|
D ].}t%          ||j                  }||                    |           /| j
                            |j        |           |D ]2}||k    r*||         j        D ]}| j
                            |           3i }|
D ]6}|                    |j        j                   t3          |j        |           7t5          ||         |
||           t          |j        |j                  }|D ]$}||v rt7          |||         |||                    %||                                         }|                                 d
|_        d|_        |                     |
          }|r2d}||                                                                          rd}|2| j	        j!        r2| j"        #                    ||         $                                           t          |j        |j                  }tK          |j        ||          }d |D             }tM          ||
|||           ||         '                                 ||         (                                 |S )zQReprocess a set of nodes within a single module.

    Return fired triggers.
    z-%s not in graph (blocking errors or deleted?)c                >    i | ]\  }}||                                 S rW   )rC   )rY   r  r   s      rM   r  z#reprocess_nodes.<locals>.<dictcomp>  s&    MMM+$4MMMrO   r   r   r6   intc                    | j         j        S ra   )r   line)r   s    rM   r(  zreprocess_nodes.<locals>.key  s     y~rO   r'  Nr      FTc                ,    h | ]}t          |          S rW   r  r  s     rM   r\   z"reprocess_nodes.<locals>.<setcomp>  s     <<<D\$''<<<rO   )r   r   r6   rC  ))rg   rm   r   find_symbol_tables_recursivefullnamer   r  r#   rj   r   r@   set_file_ignored_linesr   ignored_linesignore_errors
ignore_allset_skipped_linesskipped_linestarget_from_noder   r   clear_errors_in_targetsearly_errorsadd_error_infor   r&   r    r$   r   rz   pass_num	last_passcheck_second_passexport_types	all_typesrv   type_mapr"   update_depsverify_dependenciesr   )r9   r:   r   r?  r>   rK   	file_nodeold_symbolsold_symbols_snapshotr(  r3  r   r   rA   r   rw   r2  saved_attrsr=  new_symbolsr  checkermorenew_symbols_snapshotchangednew_triggereds                             rM   r-  r-    s      !PS\!\]]]uu	*I.y/A9?SSKMM9J9J9L9LMMMK01CY_UU    7$$$E)EmGN))	/1F1Z%JZ   N$$Y^Y5LMMMeeG    !)TY77KKN**9>7CCC  4 4Yi(5 4 4--d3333 $&K 1 1  !7888X]K0000!%	"2E5+NNN /y/A9?SSK S S;y+d"3YD@QRRR I++--GMMOOOGG$$U++D
 ((**<<>> 	D  
 # >  y!1!:!:!<!<===01CY_UU,02F G =<G<<<M 	5%w777 
)((***	)!!!rO   prefixsymbolsr   dict[str, SymbolTable]c                $   | |i}|                                 D ]v\  }}t          |j        t                    rW|j        j                            | dz             r5t          | dz   |z   |j        j                  }|                    |           w|S )a2  Find all nested symbol tables.

    Args:
        prefix: Full name prefix (used for return value keys and to filter result so that
            cross references to other modules aren't included)
        symbols: Root symbol table

    Returns a dictionary from full name to corresponding symbol table.
    r   )	r  r   r   r   rI  r8  rH  r   rv   )rf  rg  r5   r  r   rb  s         rM   rH  rH    s     gFmmoo    
ddi** 	 ty/A/L/LVVY\/Z/Z 	 /t0CTY_UUDMM$MrO   r3  list[FineGrainedDeferredNode]r   r   c                z   |D ]}|j         }||                                          }||          j        }|
J d            t          | ||||j                  }	|	                                D ]:\  }
}|                    |
t                                                    |           ;t          j
        |           d S )Nz$Tree must be processed at this stage)r   rY  r   r'   python_versionr  
setdefaultrm   rv   r,   update_protocol_deps)r   r3  r:   r>   r   r=  r   rY  r   new_depsr{   rA   s               rM   rZ  rZ  /  s      	< 	<}#,,..Y$!G-tT8W-C
 
 !) 0 0 	< 	<GWOOGSUU++227;;;;	< #D)))))rO   rw   5tuple[list[FineGrainedDeferredNode], TypeInfo | None]c                B    d fd} j         }t          |          }| |             g dfS |\  }}|r|                    d          }ng }||         }d}	d}
|D ]}t          |t                    r|}
t          |t
                    r|}	t          |t
          t          f          r	||j        vr |             g dfc S |j        |         j        rg dfc S |j        |         j        }t          |t                    r|	J |j	        k    r |             g dfS t          |	d          g}d}|j        r|}|j                                        D ]O\  }}|j        }t          |t                    r.t           dz   |z             \  }}|                    |           P||fS t          |t                     r|j        }t          |t          t
          t$          f          s |             g dfS |j	        k    r |             g dfS t          ||
          gdfS )a  Look up a target by fully-qualified name.

    The first item in the return tuple is a list of deferred nodes that
    needs to be reprocessed. If the target represents a TypeInfo corresponding
    to a protocol, return it as a second item in the return tuple, otherwise None.
    r6   r7   c                 :                          d d           d S )NzCan't find matching target for z (stale dependency?))rg   )r9   rw   s   rM   	not_foundz lookup_target.<locals>.not_foundN  s)      !_6!_!_!_`````rO   Nr   r   )r   r/   r   r   r   r   r   plugin_generatedr   rI  r   is_protocolr  r   r+  r   r   funcr   )r9   rw   rs  r   r  rZ   restr   r   fileactive_classcr5   
stale_infor  symnodemethodr[   s   ``                rM   r+  r+  D  s   a a a a a a a oG&))E}	4xLFD ZZ__


%foD DL " "dH%% 	 LdH%% 	D$8 455 	$*9L9LIKKKt8OOO :a=) 	t8OOOz!}!$!! "
 =F"" IKKKt8O)$556&*
 	J!Z--// 	& 	&MD'<D$(( &)'6C<$3FGG	f%%%z!!$	"" ydWh0ABCC  		4x} 		4x#D,778$>>rO   c                0    | j         j        dk    pt          S )Nr   )r   	verbosityr
   )r9   s    rM   ri   ri     s    ?$)?-??rO   r   &FuncDef | MypyFile | OverloadedFuncDefr   c                    t          |t                    r| |j        k    rdS | S |j        r|j        j         d|j         S |  d|j         S )a'  Return the target name corresponding to a deferred node.

    Args:
        module: Must be module id of the module that defines 'node'

    Returns the target name, or None if the node is not a valid target in the given
    module (for example, if it's actually defined in another module).
    Nr   )r   r   rI  r2  r  )rZ   r   s     rM   rP  rP    sh     $!! 	+T]""49 	+i(6649666**ty***rO   win32)z/__init__.pyz/__init__.pyiINIT_SUFFIXESz__init__.pyz__init__.pyirefresh_fileCallable[[str, str], list[str]]list[str] | Nonec                   d}||                     t                    sdS t          j                            |          }	 |                    |          }n# t          $ r g }Y nw xY w|D ]}	|	                     d          r.|	                    d          s|	                    d          dk    rG|		                    d          d         | dz   z   }
t          |
          }t          | ||           ||v r||         D ]}|                    |          }|s't          ||          }||                    |          }|r|j        |j        J  ||j        |j                  }|j        }|sJ |j        D ]}t#          |t$                    rh|j        | k    r]t'          fd|j        D                       r=|
|j        vr4|j                            |
           |j                            |
           |S )a  Look for submodules that are now suppressed in target package.

    If a submodule a.b gets added, we need to mark it as suppressed
    in modules that contain "from a import b". Previously we assumed
    that 'a.b' is not a module but a regular name.

    This is only relevant when following imports normally.

    Args:
        module: target package in which to look for submodules
        path: path of the module
        refresh_file: function that reads the AST of a module (returns error messages)

    Return a list of errors from refresh_file() if it was called. If the
    return value is None, we didn't call refresh_file().
    N)z.pyz.pyiz	__init__.r   r   r   c              3  *   K   | ]\  }}|k    V  d S ra   rW   )rY   r  r[   	shortnames      rM   rc   z0refresh_suppressed_submodules.<locals>.<genexpr>  s,      'S'SgdA	(9'S'S'S'S'S'SrO   )r   r  osr   dirnamelistdirFileNotFoundErrorr8  r  r   r+   r   r   r.   r   r_   importsr   r   anyr   suppressed_set
suppressedr   r   )rZ   r   r>   r:   r   r  ru   pkgdirentriesfnam	submoduler{   depr   
dep_moduler   impr  s                    @rM   refresh_suppressed_submodulesr    sH   0 H|4==77|tW__T""F//&))    (D (Do..	{++	 zz#!##JJsOOA&	SL9,	y)) 	64///d??G} D D		# 6!.uc!:!:J!- %		* 5 5 Dz)  %z555#/<%*#E#E :DKKK#| D D%c:66 D #& 0 0$''S'S'S'S'S'S'S$S$S !1$-U5I$I$I % 0 7 7	 B B B % 4 8 8 C C COs   A A'&A'messagec                ^    t          j        d|           }|r|                    d          S d S )Nz([^:]+):[0-9]+: (error|note): r   )rematchgroup)r  ms     rM   extract_fnam_from_messager    s1    
2G<<A wwqzz4rO   c                :    |                      dd          d         S )N:r   r   )r   )r  s    rM   "extract_possible_fnam_from_messager    s    ==a  ##rO   ru   prev_messagesc                   d}i }|D ]!}t          |          }|r||vr
|||<   |dz  }"g }d}|t          |           k     rN| |         }t          |          }|g}	||v r|dz   t          |           k     rt          | |dz                      |vrt          | |dz                      | |dz                                d          s|dz  }|	                    | |                    |dz   t          |           k     rPt          | |dz                      |vr6t          | |dz                      | |dz                                d          |                    |                    ||          |	f           |dz  }|t          |           k     Nt          |d           }g }
|D ]\  }}	|
                    |	           |
S )a6  Sort messages so that the order of files is preserved.

    An update generates messages so that the files can be in a fairly
    arbitrary order.  Preserve the order of files to avoid messages
    getting reshuffled continuously.  If there are messages in
    additional files, sort them towards the end.
    r   r   Nzmypy: c                    | d         S r$  rW   )gs    rM   r&  z5sort_messages_preserving_file_order.<locals>.<lambda>7  s
    !A$ rO   r'  )r  r   r  r8  r   r   rj   r   )ru   r  nordermsgr  groupsr   
maybe_fnamr  r5   r(  s               rM   ro   ro     s    	
AE  (-- 	D%%E$KFA F	A
c(mm

qk7<<
 AH%%6xAGGuTT-hq1uo>>F Q228<< G QXa[))) AH%%6xAGGuTT-hq1uo>>F Q228<< G
 	uyyQ//7888	Q c(mm

" F///FF  
UeMrO   )r9   r   r:   r   r   r   r6   rT   )rZ   rx   r>   r   r:   r   r6   r7   )r9   r   r:   r   r   r   r6   r7   )rZ   rx   r   rx   r9   r   r<   r   r:   r   r   rS   rR   rS   r6   r   )r   rP   r:   r   r6   r   )
r   rx   r   rx   r:   r   r9   r   r6   r7   )r   rP   r6   rP   )r:   r   r6   r   )
r   r   r   r   rI   rP   rR   rS   r6   r  )r9   r   r   r	  r   r
  r6   r   )
r9   r   r:   r   r  r
  r   r
  r6   r7   )r9   r   r:   r   r>   r   rH   r   r   r   r   r   rK   rT   r6   rP   )r9   r   r:   r   r4  r   r>   r   r   r   r6   r5  )r9   r   r:   r   r   rx   r?  r@  r>   r   rK   rT   r6   r   )rf  rx   rg  r   r6   rh  )r   rx   r3  rj  r:   r   r>   r   r   r   r6   r7   )r9   r   rw   rx   r6   rp  )r9   r   r6   rS   )rZ   rx   r   r  r6   r   )rZ   rx   r   r   r>   r   r:   r   r   r   r  r  r6   r  )r  rx   r6   r   )r  rx   r6   rx   )ru   rT   r  rT   r6   rT   )ir  
__future__r   r  r  sysr   collections.abcr   typingr   r   r   r   r2   r	   r   
mypy.buildr
   r   r   r   r   r   r   r   mypy.checkerr   mypy.errorsr   mypy.fscacher   mypy.modulefinderr   
mypy.nodesr   r   r   r   r   r   r   r   mypy.optionsr   mypy.semanal_mainr   r    mypy.server.astdiffr!   r"   r#   mypy.server.astmerger$   mypy.server.aststripr%   r&   mypy.server.depsr'   r(   mypy.server.targetr)   mypy.server.triggerr*   r+   mypy.typestater,   	mypy.utilr-   r.   r/   r0   r   r   r4   r   r   r   r   r   r   r   r   r   rf   r;   r   r   r   rl   r*  r-  rH  rZ  r+  ri   rP  platformr  sepaltsepr  r  r  ro   rW   rO   rM   <module>r     s  p p pd # " " " " " 				 				 



  $ $ $ $ $ $ 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5 5	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 1 0 0 0 0 0 $ $ $ $ $ $ ( ( ( ( ( ( ) ) ) ) ) )	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 	 !           V V V V V V V V         
 , + + + + + > > > > > > > > K K K K K K K K 0 0 0 0 0 0 : : : : : : : : % % % % % % A A A A A A A A A A     F k/ k/ k/ k/ k/ k/ k/ k/\	   88 8 8 8&: : : :.    :       J    !}!<= = = = =uE uE uE uEp   6/ / / /,   B B B B
 
 
 
-2 -2 -2 -2`. . . .0> > > >B;0 ;0 ;0 ;0|^ ^ ^ ^B   $* * * **K? K? K? K?\@ @ @ @+ + + +* <7<M<<<<< 	

	M!
	N"	M    K K K K\   $ $ $ $
+ + + + + +rO   