Stone

在上一篇“CLR探索系列之应用程序域世界”的上篇中，探讨了一些关于应用程序域在托管代码执行过程中的特性和运行机制，以及一些相关的概念。

在接下来的中篇里，就从如何实现的角度，换一个角度来探讨程序集和应用程序域是如何加载，执行。以及一些有趣的问题。

首先，有一个有趣的“鸡和蛋”的问题。我们知道，一个应用程序集里面的代码在执行的时候，首先被load，然后经过验证，接着对IL代码JIT成为本地代码才能执行。一个应用程序集只有被先加载了才能被执行，但是加载程序集的程序集，是被什么程序集加载的呢？或者，第一个程序集，是如何被加载到CLR的世界中呢？
首先，来查看一下Clix工具作为一个sscli提供的loader的main函数都做了些什么：
int __cdecl main(int argc, char **argv)

{

DWORD nExitCode = 1; // error

WCHAR* pwzCmdLine;

if ( !PAL_RegisterLibrary(L"rotor_palrt")

|| !PAL_RegisterLibrary(L"sscoree") ) {

DisplayMessageFromSystem(::GetLastError());

return 1;

}

可以看到，在clix的Main函数里面，就做了两件事情：注册Rotor的palrt模块，同时，注册sscoree模块。
在执行托管代码的库文件结构中，有三个层次：
第一层：Managed libraries
第二层：Execute Engine(CLR)
第三层：PAL
第一层里面，主要包含的是BCL；还有一些别的托管系统的库文件。例如mscorlib.dll，System.xml.dll或者是别的托管组件之类。
第二层里面，有我们非常熟悉的sscoree.dll，也就是rotor里面的托管程序的执行引擎。
在第三层PAL层里面，主要有两个文件：rotor_pal.dll，rotor_palrt.dll；在rotor的源代码解压后，clr，pal，palrt这三个文件夹是并列排列的。这也反应了这三个部分之间的关系。pal是某个特定的操作系统对PAL层的实现，而palrt是忽略操作系统的区别对PAL层的一般实现。

在 if ( !PAL_RegisterLibrary(L"rotor_palrt")|| !PAL_RegisterLibrary(L"sscoree") )这一行中，首先是加载了托管库文件结构里面最下面PAL层的针对编译好了的，一个特定的操作系统的实现。接着，又是调用加载了基于这个PAL_RT层上面的CLI的托管执行引擎：sscoree。而对于托管代码执行需要的库文件的第三层，也就是最上面一层，BCL之类的库文件的加载，则是在创建这个托管引用程序的内存结构的几个特定类型的应用程序域中加载进去的。
这样，对于托管代码执行的时候的需要的一些库文件(按照库文件的结构，从下往上)是如何加载到内存中去，以及PAL层和CLR的加载执行顺序，我们就有了一个比较清晰的认识了。
then，在注册好了PAL层和CLR之后，我们再来看看作为sscli里面提供的一个loader，是如何实现load一个exe(或许是托管的)到执行的托管进程中去的。打开Clix.app的Launch函数：
//the Launch founction of Clix.Shows how launch of first Assembly.

//launch the EE of CLI

DWORD Launch(WCHAR* pFileName, WCHAR* pCmdLine)

{

//file name

WCHAR exeFileName[MAX_PATH + 1];

DWORD dwAttrs;

//define the error type

DWORD dwError;

DWORD nExitCode;

dwAttrs = ::GetFileAttributesW(pFileName);

//省略若干对于文件名表示的文件的相关检查代码

if (dwError != ERROR_SUCCESS) {

// We can't find the file, or there's some other problem. Exit with an error.

fwprintf(stderr, L"%s: ", pFileName);

DisplayMessageFromSystem(dwError);

return 1; // error

}

//DWORD Exit Code.

//这里，调用导入进来的

nExitCode = _CorExeMain2(NULL, 0, pFileName, NULL, pCmdLine);

// _CorExeMain2 never returns with success

_ASSERTE(nExitCode != 0);

DisplayMessageFromSystem(::GetLastError());

return nExitCode;

}

首先，我们看这一句：_ASSERTE(nExitCode != 0)；程序运行到这里的时候，就是对一个托管程序的执行已经完成了，PAL，EE和相关的加载的了的BCL以及相关的托管模块和应用程序域，这些东西都已经退出内存，我们对这个加载的exe文件的执行，就到此为止了。It is the time for us to show down the lights，and went home……^_^

接着，我们再来看这一句： nExitCode = _CorExeMain2(NULL, 0, pFileName, NULL, pCmdLine)；这里，就开始执行外部导入函数了，也是经常看到的非常频繁的CorExeMain这个函数。不同的是，后面多了一个2。这是商业版本和开源版本的一点小小的区别了。

在商业版本的DotNet Framework 中，这个地方调用的函数是_CorExeMain()；可以用Dependency walker，PEID，或者是Inspect，来查看任何一个本机上面生成好了的托管的Module。查看某个Module的导入的库。

下面是我用inspect来查看一个托管模块的导入函数情况：

同时，下面是我用Dependency walker来查看MSCOREE.dll的内部函数：

这里，可以看到mscoree.dll里面包含的_CorExeMain这个函数，同时，如果是一个dll的话，就间接执行_CorDllMain这个函数。

下面，就来看看_CorExeMain这个函数都做了些什么。打开VM虚拟机目录下面的ceemain.cpp文件查看这个函数是如何实现的，都做了些什么。这个文件中包含了大部分对ee的操作，初始化，关闭等等：

//**********************************************************

// This entry point is called from the native entry piont of the loaded

// executable image. The command line arguments and other entry point data

// will be gathered here. The entry point for the user image will be found

// and handled accordingly.

//**********************************************************

__int32 STDMETHODCALLTYPE _CorExeMain2( // Executable exit code.

PBYTE pUnmappedPE, // -> memory mapped code

DWORD cUnmappedPE, // Size of memory mapped code

__in LPWSTR pImageNameIn, // -> Executable Name

__in LPWSTR pLoadersFileName, // -> Loaders Name

__in LPWSTR pCmdLine) // -> Command Line

{

// This entry point is used by clix

BOOL bRetVal = 0;

//BEGIN_ENTRYPOINT_VOIDRET;

// Before we initialize the EE, make sure we've snooped for all EE-specific

// command line arguments that might guide our startup.

//处理和文件名一起传递进来的命令参数。首先确定是不是一个托管的模块，并且对其进行一系列的检查。如果不是就直接退出托管环境的加载。

HRESULT result = CorCommandLine::SetArgvW(pCmdLine);

//把命令行缓存起来。

if (!CacheCommandLine(pCmdLine, CorCommandLine::GetArgvW(NULL))) {

LOG((LF_STARTUP, LL_INFO10, "Program exiting - CacheCommandLine failed\n"));

bRetVal = -1;

goto exit;

}

if (SUCCEEDED(result))

//如果相关的检查成功，就在这里初始化EE，调用这个文件里面的CoInitializeEE方法

result = CoInitializeEE(COINITEE_DEFAULT | COINITEE_MAIN);

if (FAILED(result)) {

VMDumpCOMErrors(result);

SetLatchedExitCode (-1);

goto exit;

}

// This is here to get the ZAPMONITOR working correctly

INSTALL_UNWIND_AND_CONTINUE_HANDLER;

// Load the executable

bRetVal = ExecuteEXE(pImageNameIn);

if (!bRetVal) {

// The only reason I've seen this type of error in the wild is bad

// metadata file format versions and inadequate error handling for

// partially signed assemblies. While this may happen during

// development, our customers should not get here. This is a back-stop

// to catch CLR bugs. If you see this, please try to find a better way

// to handle your error, like throwing an unhandled exception.

EEMessageBoxCatastrophic(IDS_EE_COREXEMAIN2_FAILED_TEXT, IDS_EE_COREXEMAIN2_FAILED_TITLE);

SetLatchedExitCode (-1);

}

UNINSTALL_UNWIND_AND_CONTINUE_HANDLER;

exit:

STRESS_LOG1(LF_STARTUP, LL_ALWAYS, "Program exiting: return code = %d", GetLatchedExitCode());

STRESS_LOG0(LF_STARTUP, LL_INFO10, "EEShutDown invoked from _CorExeMain2");

EEPolicy::HandleExitProcess();

//END_ENTRYPOINT_VOIDRET;

return bRetVal;

}

这里，就完成了对一个exe文件的加载过程。同时，在bRetVal = ExecuteEXE(pImageNameIn)；这一行也调用了执行这个文件的方法。继续查看这个方法的实现：

BOOL STDMETHODCALLTYPE ExecuteEXE(HMODULE hMod)

{

STATIC_CONTRACT_GC_TRIGGERS;

_ASSERTE(hMod);

if (!hMod)

return FALSE;

ETWTraceStartup::TraceEvent(ETW_TYPE_STARTUP_EXEC_EXE);

TIMELINE_START(STARTUP, ("ExecuteExe"));

EX_TRY_NOCATCH

{

// Executables are part of the system domain

SystemDomain::ExecuteMainMethod(hMod);

}

EX_END_NOCATCH;

ETWTraceStartup::TraceEvent(ETW_TYPE_STARTUP_EXEC_EXE+1);

TIMELINE_END(STARTUP, ("ExecuteExe"));

return TRUE;

}

这里，终于找到了我们需要找的东西，调用了应用程序域里面的执行Main函数的方法，接着打开Assembly.Cpp文件里面的这个方法，查看这个方法是如何实现在一个应用程序域里面执行一个新加载的Module的Main函数的：

INT32 Assembly::ExecuteMainMethod(PTRARRAYREF *stringArgs)

{

………………..

BEGIN_ENTRYPOINT_THROWS;

Thread *pThread = GetThread();

MethodDesc *pMeth;

{

// This thread looks like it wandered in -- but actually we rely on it to keep the process alive.

pThread->SetBackground(FALSE);

GCX_COOP();

pMeth = GetEntryPoint();

if (pMeth) {

RunMainPre();

hr = ClassLoader::RunMain(pMeth, 1, &iRetVal, stringArgs);

}

}

//省略执行结束的销毁相关内容的执行逻辑

return iRetVal;

}

到这里，找到了最后执行一个load了的模块的Main方法的地方，是在ClassLoader里面的RunMain方法中。而上面的ExecuteMainMethod方法，只是为Module的执行提供了一个从应用程序域的角度来控制的环境，为已经加载了的一个模块的执行分配一个线程，同时，处理这个模块执行好了之后相关的操作。

我们就接着追踪最后RunMain最后都干了些啥，最后一段代码，也是vm虚拟机目录下面的clsload.cpp这个文件里面的方法，（从这里，我们也看到了Rotor中非常好的层次设计和架构设计，每一层的事情和相关的处理逻辑，都控制相关的层面上面，绝不在上面一层做下面的一层的事情）：

/* static */

HRESULT ClassLoader::RunMain(MethodDesc *pFD ,

short numSkipArgs,

INT32 *piRetVal,

PTRARRAYREF *stringArgs /*=NULL*/)

{

STATIC_CONTRACT_THROWS;

_ASSERTE(piRetVal);

DWORD cCommandArgs = 0; // count of args on command line

DWORD arg = 0;

LPWSTR *wzArgs = NULL; // command line args

HRESULT hr = S_OK;

*piRetVal = -1;

// The exit code for the process is communicated in one of two ways. If the

// entrypoint returns an 'int' we take that. Otherwise we take a latched

// process exit code. This can be modified by the app via setting

// Environment's ExitCode property.

//设置返回code的类型

if (stringArgs == NULL)

SetLatchedExitCode(0);

//pFD这个指针是指向的每个在内存里面的实例的instance data的方法列表，也就是一个叫做ObjHeader的指针。我们在深入研究System.Object在内存里面布局的时候，会看到这个东西。对于每个在内存中的instance，实例的相关数据在内存中开始的第一个地址的前一个位置，保存的是一个指向这个MethodTable方法列表的指针。这个方法列表，是保存在EE的私有内存地址中的，用来方便对执行的时候的实例和对象的控制。而这个指针的前一个指针，则表示的是一个叫做SyncBlock table的东西，也是用于EE对对象的控制的。而一个实例的数据，是保存在GC堆里面的。

//下面一句的用处，就是如果这个指向这个实例的方法的指针是空的时候，（一个对象的方法可以为空，但是指向这个对象的实例的method table的指针不能为空），就会提示错误。

if (!pFD) {

_ASSERTE(!"Must have a function to call!");

return E_FAIL;

}

CorEntryPointType EntryType = EntryManagedMain;

ValidateMainMethod(pFD, &EntryType);

if ((EntryType == EntryManagedMain) &&

(stringArgs == NULL)) {

// If you look at the DIFF on this code then you will see a major change which is that we

// no longer accept all the different types of data arguments to main. We now only accept

// an array of strings.

wzArgs = CorCommandLine::GetArgvW(&cCommandArgs);

// In the WindowsCE case where the app has additional args the count will come back zero.

if (cCommandArgs > 0) {

if (!wzArgs)

return E_INVALIDARG;

}

}

ETWTraceStartup::TraceEvent(ETW_TYPE_STARTUP_MAIN);

TIMELINE_START(STARTUP, ("RunMain"));

EX_TRY_NOCATCH

{

MethodDescCallSite threadStart(pFD);

PTRARRAYREF StrArgArray = NULL;

GCPROTECT_BEGIN(StrArgArray);

// Build the parameter array and invoke the method.

//分为两种情况来处理：有参数和没有参数

if (EntryType == EntryManagedMain) {

if (stringArgs == NULL) {

// Allocate a COM Array object with enough slots for cCommandArgs - 1

StrArgArray = (PTRARRAYREF) AllocateObjectArray((cCommandArgs - numSkipArgs), g_pStringClass);

// Create Stringrefs for each of the args

for( arg = numSkipArgs; arg < cCommandArgs; arg++) {

STRINGREF sref = COMString::NewString(wzArgs[arg]);

StrArgArray->SetAt(arg-numSkipArgs, (OBJECTREF) sref);

}

}

else

StrArgArray = *stringArgs;

}

#ifdef STRESS_THREAD

OBJECTHANDLE argHandle = (StrArgArray != NULL) ? CreateGlobalStrongHandle (StrArgArray) : NULL;

Stress_Thread_Param Param = {pFD, argHandle, numSkipArgs, EntryType, 0};

Stress_Thread_Start (&Param);

#endif

ARG_SLOT stackVar = ObjToArgSlot(StrArgArray);

if (pFD->IsVoid())

{

// Set the return value to 0 instead of returning random junk

*piRetVal = 0;

threadStart.Call(&stackVar);

}

else

{

*piRetVal = (INT32)threadStart.Call_RetArgSlot(&stackVar);

if (stringArgs == NULL)

{

SetLatchedExitCode(*piRetVal);

}

}

GCPROTECT_END();

fflush(stdout);

fflush(stderr);

}

EX_END_NOCATCH

ETWTraceStartup::TraceEvent(ETW_TYPE_STARTUP_MAIN+1);

TIMELINE_END(STARTUP, ("RunMain"));

return hr;

}

在这个方法中，还设计到了一系列对COM口的交互，其中每一行，都需要对托管应用程序在内存中的结构有个清晰的了解，对这个方法做深入的分析，以及执行的流程，就是下一篇博文的事情了。^_^

Ps：分析追踪了大概6，7个文件，从CLI,到CLR再到应用程序域,然后到ClassLoad里面的方法,终于在一定层次上面搞清楚了一个托管应用程序的加载过程，以及这个过程中CLI，EE，AppDomain的加载执行过程和顺序.终于从程序的加载，刨到了应用程序域的世界。这个和我的初衷，写一篇从代码分析应用程序域似乎有些不符合…

为此，就改一个题目，改成系列中的中篇吧…

另外，我以前是写了一半保存在blog上面的，后来中途写的时候，没保存好，丢失了，又重新写了一遍…不过灵机一动，在baidu里面搜索我的文章的题目，在快照里面找到了我以前的一个版本，^_^，以后都在word里面先写好了再整过来。

文章里面有纰漏的地方，欢迎大家指正！:)
后记:
补充说明下,一个托管对象在内存里面的格式:

托管对象的结构如下：

m_SyncBlockValue

对象指针－>  m_pMethodTable

Data

在每个托管对象的开始是该对象类型的方法表。在方法表之前是m_SyncBlockValue。

m_SyncBlockValue的高6位用来标记m_SyncBlockValue的用途。SyncBlockValue的低26位用来存储哈希码，SyncBlock索引或SpinBlock。

低26位值的含义由高6位来决定。

在CLR的世界中，有一系列的令人Amazing的技术和架构。其中，CLR对应用程序在内存中内存分配，执行模型，程序之间的交互等一系列的技术，值得每一个致力于DotNet平台的技术人员深究。
  编程人员在开发的过程中，如果把程序集的加载(Assemblies Load)，反射(Reflection)，寄宿(Hosting)，应用程序域(AppDomain)，这四种技术结合起来使用的话，不仅能更好的使用CLR这个平台提供的强大的功能，而且能够构建更安全，更健壮的应用程序代码。
  这篇博文里，就是使用托管代码的动态调试工具，来研究一下CLR内部AppDomain的世界。
  首先，从一个C#程序开始：
   class Program
   {
   static void Main(string[] args)
   {
   Program b = new Program();
   b.test();
   System.Console.ReadLine();
   }
   public void test()
   {
   int i = 67;
   System.Console.WriteLine((char)i);
   System.Console.WriteLine((char)67);
   i = 1;
   }
   }
  运行了这个应用程序以后，我们打开windbg，attach到这个托管进程。
  .load SOS
  加载SOS扩展调试模块，可以使用.chain指令查看加载是否正确。
  0:003> lm
  start end module name
  00400000 00408000 TestConcoleApp (deferred)
  76990000 76acd000 ole32 (deferred)
  77be0000 77c38000 msvcrt (deferred)
  省略若干
  77fc0000 77fd1000 Secur32 (deferred)
  78130000 781cb000 MSVCR80 (deferred)
  79000000 79045000 mscoree (deferred)
  79060000 790b3000 mscorjit (deferred)
  790c0000 79b90000 mscorlib_ni (deferred)
  79e70000 7a3d6000 mscorwks (deferred)
  查看下意境加载了的模块，然后使用ld命令把我们的调试符号文件载入。VS在编译生成一个Console App的时候，在debug模式的时候会在debug的bin目录下生成一个和应用程序同名的pdb文件。我们要做的，就是载入这个文件：
  0:003> ld TestConcoleApp
  *** WARNING: Unable to verify checksum for G:\Projects\TestConcoleApp\TestConcoleApp\bin\Debug\TestConcoleApp.exe
  Symbols loaded for TestConcoleApp
  有一个警告，咋这里先不管，再使用lm查看已经加载了的模块的时候，可以看到这个module的调试符号文件已经被加载上了。
  此时，我们可以查看下Excute Engine (CLR)的堆里面都有些什么东西，我们可以使用!EEHeap命令，EE的意思，就是CLI的执行引擎，也就是我们常说的CLR。这个命令可以查看到一个托管进程里面的garbage-collected 和 Loader heaps相关信息。
  0:003> !eeheap
  PDB symbol for mscorwks.dll not loaded
  Loader Heap:
  --------------------------------------
  System Domain: 7a38f918
  LowFrequencyHeap: Size: 0x0(0)bytes.
  HighFrequencyHeap: 00a62000(8000:1000) Size: 0x1000(4096)bytes.
  StubHeap: 00a6a000(2000:1000) Size: 0x1000(4096)bytes.
  Virtual Call Stub Heap:
   IndcellHeap: Size: 0x0(0)bytes.
   LookupHeap: Size: 0x0(0)bytes.
   ResolveHeap: Size: 0x0(0)bytes.
   DispatchHeap: Size: 0x0(0)bytes.
   CacheEntryHeap: Size: 0x0(0)bytes.
  Total size: 0x2000(8192)bytes
  /**********************************************
  Loader Heap 中的系统域。这个域和下面的Shared Domian一起，是对托管的宿主程序，以及托管代码不可见的。这个域加载了两个CLR执行中十分重要的Module，MSCorEE.dll和MScorwks.dll。MSCorEE.dll这个文件就是大家熟悉的shim，垫片。在CLR加载中起到的重要作用。这里就不分析了，大家可以参考别的文献的介绍。
  对于每个应用程序域，都会有自己的安全描述符，安全上下文以及默认的上下文。这三个部分可以支持一个应用程序域来自定义一个单独实施的安全策略，譬如，可以用来确保宿主程序在加载托管代码的时候不会对这些重要的数据结构造成破坏。
  **********************************************/
  --------------------------------------
  Shared Domain: 7a38fef0
  LowFrequencyHeap: 00a90000(2000:1000) Size: 0x1000(4096)bytes.
  HighFrequencyHeap: Size: 0x0(0)bytes.
  StubHeap: 00a9a000(2000:1000) Size: 0x1000(4096)bytes.
  Virtual Call Stub Heap:
   IndcellHeap: Size: 0x0(0)bytes.
   LookupHeap: Size: 0x0(0)bytes.
   ResolveHeap: 00aab000(5000:1000) Size: 0x1000(4096)bytes.
   DispatchHeap: 00aa7000(4000:1000) Size: 0x1000(4096)bytes.
   CacheEntryHeap: Size: 0x0(0)bytes.
  Total size: 0x4000(16384)bytes
  /**********************************************
  在共享域中，加载所有的应用程序域中都要使用到的assemblies，譬如MScorlib.dll。装载了System.Object，System.ValueType这样的基础类。
  **********************************************/
  --------------------------------------
  Domain 1: 154250
  LowFrequencyHeap: 00a70000(2000:2000) Size: 0x2000(8192)bytes.
  HighFrequencyHeap: 00a72000(8000:2000) Size: 0x2000(8192)bytes.
  StubHeap: Size: 0x0(0)bytes.
  Virtual Call Stub Heap:
   IndcellHeap: Size: 0x0(0)bytes.
   LookupHeap: Size: 0x0(0)bytes.
   ResolveHeap: Size: 0x0(0)bytes.
   DispatchHeap: Size: 0x0(0)bytes.
   CacheEntryHeap: Size: 0x0(0)bytes.
  Total size: 0x4000(16384)bytes
  /**********************************************
  对于特定的寄宿程序，可以根据需要创建多个应用的默认域。例如IE，ASP.Net，或者是SqlServer，可以创建一个或者是多个默认的域。域名默认情况下的name就是module的名称。
  在默认域中，应用程序执行的时候需要装载经来的assemblies可以被加载到这里。
  在每个应用程序域中，代码创建的对象不能直接访问另外的应用程序域中的代码。如果要访问这些代码，可以采用静态的委托，或者是appDomain的自己的方法来实现。
  **********************************************/
  --------------------------------------
  Jit code heap:
  LoaderCodeHeap: 00db0000(10000:1000) Size: 0x1000(4096)bytes.
  Total size: 0x1000(4096)bytes
  对于托管的应用程序，有两种把IL代码编译成本地代码的方式。一种是在第一次运行的时候，调用JIT模块来实时编译，编译好了的本地代码就放到这里。在PE文件中的相应的代码处，就用一个指针指引CLR到这里来找相关的编译好了的本地代码。第二中是安装的时候就编译成为本地代码。
  同时可以看到，Jit Heap占用很少的内存空间。
  --------------------------------------
  Module Thunk heaps:
  Module 790c2000: Size: 0x0(0)bytes.
  Module 00a72c24: Size: 0x0(0)bytes.
  Total size: 0x0(0)bytes
  --------------------------------------
  Module Lookup Table heaps:
  Module 790c2000: Size: 0x0(0)bytes.
  Module 00a72c24: Size: 0x0(0)bytes.
  Total size: 0x0(0)bytes
  --------------------------------------
  Total LoaderHeap size: 0xb000(45056)bytes
  总共的loader Heap的大小大概在45kb左右。
  =======================================
  Number of GC Heaps: 1
  generation 0 starts at 0x013b1018
  generation 1 starts at 0x013b100c
  generation 2 starts at 0x013b1000
  ephemeral segment allocation context: none
  下面的这两个segment对于应用程序域的其他代码来说是read only的，所以，这两部分的空间比较小。这块经常保存的是小的segement片段。除非你是很长很长的字符串。而大的object，则保持在LOH中。GC Heap，可以有多个。每个GC Heap中，都可以有一个LOH。
  而每个GC Heap的总大小＝Segment占用的空间+LOH
   segment begin allocated size
  0014d720 790d5588 790f4b38 0x0001f5b0(128432)
  013b0000 013b1000 013b3ff4 0x00002ff4(12276)
  Large object heap starts at 0x023b1000
   segment begin allocated size
  023b0000 023b1000 023b3250 0x00002250(8784)
  Total Size 0x247f4(149492)
  ------------------------------
  GC Heap Size 0x247f4(149492)
  总共的GC堆大概150kb。
  ---------------------------------------------------------------------------------------------------------
  托管线程的内存结构：
  这里，简单的交代一下一个托管进程的内存结构。在创建了一个托管的应用程序的线程以后，首先默认情况下创建了最少3个应用程序域，就是系统域，共享域，和默认域。前两个对托管的用户代码来说是不可见的。当时，可以调用共享与中的assemblies。用户的托管代码，和模块被load到默认域中。一个托管的宿主可以根据需要创建一个或者是多个默认域。
  托管进程，线程(hard thread，soft thread)，应用程序域，程序集，模块的关系
  托管进程，应用程序域，程序集，模块，这四个概念从左到右是一对多的关系。及一个托管可以对应多个应用程序域，一个appdomain可以对应多个assemblies，一个assembly可以对应多个modules。modules就是我们经常看的.exe或者是.dll。exe文件是windows下对PE文件格式扩展了的托管模块。modules也可以是托管的动态链接库文件。
  对于线程，情况有点特殊。这里，首先要区别一个概念，操作系统的进程创建的线程和System.Threading.Thread这个类表示的线程。这里，把操作系统创建的进程叫做hard thread，System.Threading.Thread这个类表示的线程叫做soft Thread。hard Thread和应用程序域的关系，是多堆多的关系。就是一个应用程序域中可以存在多个hard Thread。而一个hard Thread，也可以存在于多个应用程序域里面。而soft Thread，是由应用程序域中的assemblies创建，所以，它只存在于相应的应用程序域中。
  当一个系统的hard thread进入某个应用程序域中进行操作的时候，这个应用程序域就会实例化一个System.Threading.Thread类来完成这个线程对应的工作。
  应用程序域的环境变量属性：
  对于每个应用程序域，有一系列的Environment属性可以设置，通过设置这些属性，可以配置一个应用程序域的特性，来满足各种对于安全，性能等许多方面特别的需求。
  可以参考MSDN的这里：
  http://msdn2.microsoft.com/en-us/library/system.appdomain_properties.ASPx
  获知应用程序域相关的所有的Properties。
  特别说明：应用程序域的动态目录：
  对于应用程序域的所有的属性，需要特别提到一个叫做DynamicDirectory 的属性。
  这个属性的生成，有两个部分，譬如我本机上面的一个ASP.net宿主进程的缓存文件夹：
  C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\mesapplication\8a8504fd\e1680364
  C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files\mesapplication这一部分，由一个叫做DYNAMIC_BASE的属性来定义。同时包含了这个应用程序域中加载的项目文件信息。
  后面的两部分8a8504fd\e1680364，根据项目文件的不同，由应用程序的APP_NAME这个属性来决定。
  这样何在一起，就构成了在调试的过程中，进程在本机的缓存文件夹。
  在接下来的下篇中，讲从如何实现的角度，以实例源代码来分析AppDomain的运行机制。

A few days ago, some of us in my group were discussing about the GC mechanism used in the CLR. It was an interesting discussion, and like most discussions, inconclusive. So I decided to put (almost) everything interesting to know about the GC on paper. Some of the people who read this, particularly Vinod – he is sick of my long mails not converting into articles - suggested that I share this with a broader audience and put it on the blog.
While writing, I was actually vacillating – more than enough has been written about GC, so why write another piece? But then, I think no single article captures everything, and since there are so many anyway, why not another one?? :-) As far as I know, there is nothing new here – everything has already been said by the likes of Patrick Dussud, Brian Harry, Chris Brumme, Maoni Stephens, Rico Mariani, Jeff Richter – this is just a compilation, and not a very comprehensive one at that, esp. on the Best Practices, so I have included a Further Reading section. For whatever it is worth, here it is!!
1. The Problem Domain – Memory Management
1) Applications composed of components / libraries written by different developers at different points in time
2) These components have no idea of resources are allocated / released by each other – they are only aware of how they manage resources themselves
3) The substrate needs to provide a common memory management infrastructure which would let these components work with each other
4) The mechanism used should
* Be optimal on memory utilization
* Provide strong object locality to take get cache efficiency and take advantage of CPU Registers & Caches
* Ensure code correctness
* Minimize the burden on the developer
* Support scenarios like Finalization, Weak References and Pinning
2. Memory Management – What are the Choices?

2.1 Reference Counting
Each component keeps a count of number of clients using it and destroys itself when the last client Releases it
* Example: COM, Number of large-scale C++ projects
* Depends on components and clients keeping a correct count, mistakes can lead to a memory leak (accumulation of unreferenced objects) or premature releasing
* Difficult to resolve circular references
2.2 Garbage Collection - Mark and Sweep
* Example: C Runtime Heap
* Initially, allocate objects on the heap sequentially
* At some stage, mark the objects that are dead and can be removed
* Free the dead object slots at some stage
1) In C/C++ the programmer has to do it since the runtime can be misled about object size by type-casting, void pointers, etc.
2) Environments like CLR where type safety is guaranteed are always aware of object size and can free memory on their own
* Maintain a list of free slots (Free List)
* On next allocation request traverse the list and give the smallest possible slot that can fulfill the memory allocation request
* Advantage: Minimal house-keeping overhead (just one freelist)
* Disadvantage: Every allocation request requires a walk thru the freelist, makes allocations slow
* Disadvantage: Heap fragmentation – there may not be a single contiguous block to accommodate a request for a large block
2.3 Garbage Collection – Copy and Collect
* Keep two heaps
* Allocate only from one heap
* When collection is triggered on the heap, copy all alive objects to the second heap
* Switch the roles of heaps
* Advantage: Conceptual Simplicity
* Disadvantage: Copy operation needs to be done for all objects
* Disadvantage: Blocks a lot of memory unneccessarily
2.4 Garbage Collection – Mark and Compact
* Example: CLR, some JVM implementations
* Same as Mark and Sweep except that as free space from dead objects is claimed, the alive objects are compacted together to leave a single contiguous block of free memory
* Disadvantage: Compacting memory can be expensive for large objects
* Advantage: Allocation requires just a pointer to keep track of the top of the heap, so it is lightning fast
* Advantage: No heap fragmentation
3. What Happens in the CLR

3.1 Marking, Sweeping and Compacting

3.1.1 Large and Small Objects
There are actually two GC heaps: one for large objects and another for small objects. Large objects are kept in a separate heap called the Large Object Heap (LOH) which is Mark and Swept since copying large objects causes extra overhead
* The definition of large object used to be >= 20000 bytes in v1.0
* It changed to >= 85,000 bytes in 1.1
* It may change in future, so don’t make hard assumptions on the size
Smaller objects are kept in multiple heaps which are compacted from time to time.
3.1.2 Marking a heap (all GC heaps)
* GC assumes that all heap is garbage.
* It then starts building a graph of all live objects starting from the application’ roots.
* Roots are references to live objects. They exist in:
1) The call stack of a thread
2) Static and global objects
3) Freachable queue (see Finalization)
4) Places like the CPU registers
* The JIT compiler tracks the roots and gives a list to the GC
* All objects that are reachable are considered alive, everything else is dead and can be collected
3.1.3 Sweeping a heap (all GC Heaps)
* Merge adjacent freed blocks of memory into one.
* Maintain a FreeList of all space that is free.
* Allocation would now need to walk the FreeList
3.1.4 Compacting a heap (only the small object heaps)
* Slide objects so that all alive objects get compacted into one contiguous block
* Update references to each object so that they now point to new memory locations
* Move the heap pointer (pointer to the next object location) to the top of the heap
* Allocation would now happen at the heap pointer
* Each allocation moves the pointer further up
* LOH is only swept and never compacted
* Small Object heaps are frequently swept and less frequently compacted
3.2 Generational GC

3.2.1 Memory Usage Observations
Generational GC or ephemeral GC is an optimization based on the following observations:
* Newer objects tend to live for a short duration
* Older objects tend to live for longer durations
* Newer objects have strong relationships with each other and frequently accessed around the same time
3.2.2 The Three Generations
The GC keeps small objects in three generations:
* Gen 0: Newly created objects which have not seen a collection yet
* Gen 1: Objects which have survived one collection
* Gen 2: Objects that have survived two collections
* Gen 0 and Gen 1 are also called ephemeral generations, and they always live together in a single segment called the ephemeral segment (see Segments later)
* GC tries to fit Gen 0 in the L2 Cache of the CPU
3.2.3 Collections by Generations
* Every time Gen 0 heap crosses a threshold (called a “budget” – see allocation and collection below), a GC is triggered
* Gen 0 is collected very frequently.
* Frequency of Gen 0 collections > Gen 1 Collections > Gen 2 collections (Gen 1: Gen 2 being 10 : 1 is healthy)
* Collection for Gen N collects all Gens < N, so
1) A Gen 1 collection also causes a Gen 0 collection.
2) A Gen 2 collection also causes a Gen 1 and Gen 0 collection. It also causes a LOH Collection. This is called a full collection
* If memory pressure is high, GC is more aggressive in collection and Gen 1 and Gen 2 collections may happen more frequently.
3.2.4 Generational GC Benefits
* No need to compact a single gigantic heap – multiple smaller heaps of different sizes are compacted at different frequencies – this is faster
* Objects which have a similar lifespan stay closer together. This improves object locality. Object locality helps improve perf with modern CPUs where proc speed >> memory speed
* The time required in the Mark phase can be significantly reduced by ignoring the inner references of older generation objects
* However, an older object may have been written to and may have hold a reference to a newer object
* To track this, the GC maintains an internal structure called the “card table”
1) Card Table consists of entries: 1 bit for 128 bytes of memory. This means one DWORD corresponds to 4k, which incidentally is the size of a page
2) So you can think about the card table as an array of DWORDs with every entry referring to a page and further split into every bit tracking a 128 byte range
3) Every time an object gets modified, the JIT emits code to write to the object as well as the card table
a) This is possible because of what is called the “Write barrier” – the JIT preventing a write directly
b) Another approach is to use the GetWriteWatch Win32 API that tells you what has changed
4) To map an object to the address range, another internal look up dictionary like structure is used.
3.3 Segments
GC reserves memory in segments. A segment is a unit of reservation.
3.3.1 Generations and Segments
* To begin with, GC creates two segments – one for Gen 0 and Gen 1, and the other for LOH
* Gen 0 and Gen 1 share a single segment throughout the lifecycle of the program. This is called the ephemeral segment.
* Gen 2 may have zero or more segments of its own
* LOH has one or more segments of its own
3.3.2 Segment Size
* As of CLR v2, each segment typically = 16 MB (but not fixed, dynamically tuned)
* Each Heap Segment has a Reserved Size and a Committed Size. Committed <= Reserved
* Initially reserve, commit on allocation on demand
3.3.3 Heap Expansion and Contraction
A heap expands or contracts by adding / deleting heap segments. A segment is allocated whenever there is a need for more memory. This can happen if
1) Existing LOH segments cannot satisfy an object allocation request. A new LOH segment is created.
2) Objects getting promoted from Gen 1 during a Mark Phase cannot be accommodated any more in the existing Gen 2 segments. A new Gen 2 segment is created.
3.3.3.1 Segment Allocation
* To allocate a new segment, GC calls VirtualAlloc.
* If there is no free contiguous block in the heap large enough for a segment
1) Segment Allocation fails
2) The allocation request also fails
3) GC returns NULL to the Execution Engine (EE)
4) EE throws an OOM exception
3.3.3.2 Segment Deletion
A segment which is not in use, is deleted in a full collection and memory is returned to the OS
* This can be prevented with a feature introduced with CLR 2.0 called VM Hoarding which can be turned on via hosting APIs
* In fact with hosting, the CLR host can do pretty much anything. As an example, SQL Server may not want to report the memory to CLR in a transparent way to control its behavior, or fail allocations, or reserve / commit memory in advance, etc. There is a lot to memory control for a CLR host, but a full discussion of hosting is outside the scope of this article.
3.3.4 Relationship between Heaps, Segments and Generations
1) To a program, the CLR exposes a single contiguous heap
* For MP machines, when Server GC is turned on, there is a full heap and a dedicated GC thread per CPU, but to the app, there is still only one heap across all CPUs (see GC Flavors later)
2) Segments are units in which the GC internally reserves and commits virtual memory, so a heap consists of multiple segments
3) Generations are object lifetime groupings and are used by the GC to trigger collections for small objects, so logically, they orthogonal to segments. Physically:
* Gen 0 and Gen 1 always live in the same segment called the ephemeral segment
* Gen 2 lives in zero or more segments
* LOH lives in one or more segments
3.4 Allocation and Collection
At a macro level, during allocation, depending on the object size:
* Small object (< 85000 bytes) - object gets created on ephemeral segment in Gen 0, - just move the Heap pointer forward
* Large object (>= 85000 bytes) – object gets created on LOH (no co-relation with Generations), so scan the FreeList for a large enough slot, or allocate a new LOH segment
3.4.1 Allocation Context
* As mentioned earlier, memory is not allocated upfront, memory is only reserved at a segment level
* For optimization, the Allocator doles out memory in 8 kb chunks – each chunk is called an Allocation Context
* An allocation context belongs to a thread
* Allocation of each chunk requires a lock for a UP machine (GC uses lightweight spin locks) and is lock free on MP machines (see GC flavors later)
3.4.2 Zeroing the Memory
When a chunk is given, GC zeroes the memory. This is done for two reasons:
1) Security – you do not stumble upon left over garbage
2) It helps with Fail Fast. Code referencing a null pointer fails faster than code referencing some random data
3.4.3 Allocating Memory to an Object
When it comes to assigning memory to an object
* If it is within the current allocation context, the GC just needs to move the pointer
* If it is outside the allocation context, the GC needs to allocate another allocation context
3.4.4 Generation Budget
For each generation, the GC keeps a “budget” – a threshold value which when crossed triggers a collection in that generation
* When the GC needs to come up with an allocation context and realizes it has crossed the generation budget, a collection is triggered for that generation.
* For Gen 0, the budget is much smaller than the ephemeral segment
* The budget is dynamically adjusted by the GC over the lifetime of an app
3.5 Finalization

3.5.1 The Finalization Problem
Objects that need to explicitly release resources as a part of their cleanup (typically unmanaged resources like database connections, file handles, etc.), cannot rely on GC alone to do a cleanup.
* It is desired that the calling code may do the cleanup imperatively
* It is also desired that during collection, the object gets a chance to clean itself up if the calling code has not done so imperatively earlier
3.5.2 Finalization in CLR

3.5.2.1 Imperative Finalization
The calling code can call some specific method (typically called “Close” or “Dispose”) and cleanup the object imperatively. Types are supposed to implement the IDispose interface for this (see IDispose later)
3.5.2.2 Finalization by the Runtime
To indicate to the GC that there is a cleanup required, the object can implement a method called Finalize
* The method called Finalize is exposed as a destructor in C#.
* A destructor in C# is not really a destructor, it just expands to a method called Finalize which in turn calls the base class’ Finalize method
3.5.2.3 Finalization Mechanism
The system keeps a track of objects with a Finalize method in a structure called the Finalization queue. During the mark phase, after the graph of reachable objects has been built, the Finalization queue is checked, and if there is an object in the Finalization queue that is not on the graph:
1) GC puts it on a separate queue called the F-Reachable queue, and adds the object to the graph of reachable objects
* Since the object is now reachable, it does not get collected
* Since this object survives a collection it gets promoted to a higher generation.
* The same happens to all the objects referenced by this object
2) At some stage, GC invokes a special runtime thread to call Finalize on each object in the Freachable queue
* No guarantees on when Finalize would be called
* No specific order in which objects are finalized
* Exception in a Finalize method is interpreted by the GC as a proper method exit and not an exception
3) Finalized objects are removed from the Freachable queue and become unreachable and would be collected in the next collection
* So a Finalizable object and each object that it references, take two collections in a higher generation than otherwise
* If an object has been imperatively cleaned up, this overhead should be avoided
* Therefore a method called GC.SuppressFinalize is provided which should be called by an object if it has been cleaned up imperatively to avoid being put on the FReachable queue. This also prevents a Finalizable object from being promoted to the next gen (unless already in Gen 2)
3.5.3 Resurrection
* Since a Finalizable object after dying becomes alive again when put on the Freachable list, the phenomenon is called Resurrection. This is shortlived since the GC collects it shortly there after
* However, Resurrection can be more permanent if in the Finalize method, the object puts a pointer to itself in a global / static variable. Now the object is reachable even after being removed from the Freachable queue, so are the other Finalized objects to which this object may have held references.
* The resurrected object now needs to be put back on the Finalization list so that when it eventually gets collected, it gets Finalized. For this, the object should call GC.ReRegisterForFinalize()
3.5.4 Implementing a Finalizable Object

3.5.4.1 Imperative Cleanup
The mechanism an object uses to enable imperative cleanup is called the “Dispose Pattern”. The Dispose pattern is realized by implementing the IDisposable interface, which looks as follows:
public interface IDisposable
{
void Dispose();
void Dispose(bool disposing);
}
* Typically the object would keep track of whether Dispose has been called or not by using a private flag
* The Dispose() method maybe called imperatively. Typically Dispose() calls Dispose(bool) with a value of true and then calls GC.SuppressFinalize(this)
* Depending on what value is passed to the Dispose(bool), there are two possibilities:
1) Dispose(true) is called by the user code directly / indirectly. Both managed and unmanaged resources can be cleaned up here
2) Dispose(false) is called by the runtime from inside the Finalizer. Here, since the managed resources of the object may have already been cleaned up by the GC, no managed cleanup is done, and only unmanaged resources are cleaned up.
3.5.4.2 Runtime Cleanup
To enable the runtime to cleanup the code, the Finalize method needs to be implemented. This is typically combined with implementing the Dispose pattern. Here’s sample code for writing a Finalizable Type (taken from the MSDN documentation)

class Base : IDisposable
{
 private bool disposed = false;

 ~Base() // expands to Finalize, called by runtime
 { Dispose(false);  }


 public void Dispose() // called by user code
 { 
  Dispose(true); 
  GC.SuppressFinalize(this);
 } 


 public void Dispose(bool IsDisposing)
 {
  if (disposed) return;

  if (IsDisposing)
  { 
   // clean up managed as well as unmanaged resources
  }
  else
  {
   // clean up unmanaged resources only 
  }
       
  disposed = true;
 }
    
       
 // some objects (like Database Connections, Files, etc.) prefer Close over Dispose
 public void Close() { Dispose(); }

 void DoSomething()
 {
  If (disposed) throw new ObjectDisposedException();
  // rest of the code for the method
 }

    // more code for the class

}

class Derived : Base
{
 protected override void Dispose(bool IsDisposing)
 {
  if (disposed) return;

  try         {
   if (Isdisposing)
   {
    // cleanup managed & unmanaged resources added by this class only
   }
   else
   {
    // cleanup only unmanaged resources added by this class only
   }
  }
  finally
  {
   base.Dispose(IsDisposing);
  }
 }
       
 // more code for the class
}
        

 LargeObject lo = new LargeObject(); // lo is a strong reference

 // use lo

 WeakReference wr = new WeakReference(lo, false) // not tracking resurrection
 lo = null; // remove strong reference

 // do some work

 // need lo again
 if (wr.IsAlive)
 {                 lo = (LargeObject)wr.Target; // obtain strong reference
 }

 if (lo == null)  lo = new LargeObject();

 // use lo
        

今天 好无聊啊   杀了我吧

令人震惊的作品

继续

今天把以前看到得几个很猛的设计作品 传了上来