Session DNA08: Advanced Visual FoxPro Servers

Session: DNA08

Advanced Visual Foxpro Servers

By Calvin Hsia

Overview

Each release of Visual FoxPro has improved application interoperability. The heart of this capability is COM. We will discuss the implementation and design considerations for some of the ways VFP COM servers can be created. Examples of publishing VFP interfaces to VB, Excel and VC will be demonstrated. Multiple threading will be discussed extensively, from a beginner's perspective. Examples of multi-threading COM servers will be shown, with emphasis on Web deployment. Practical applications of multi-threading will be discussed.

Building Servers:

One of the main goals of COM was the ability to create software components in any language that could communicate intelligently (often called Automation) with each other. In the DOS days, an application developer could not take advantage of pre-existing software modules: most parts had to be written from scratch. If the DOS days programmer wanted to take advantage of Excel's built in mortgage calculation functions, he had to write them himself. With COM Automation, modern day applications can consist of a variety of software components. A law office application can use Automation to remote control Microsoft Word to format fancy legal documents, control Excel to display bar charts or calculate financial functions, and automate a Visual Foxpro COM server to manipulate the client data base.

A scenario that is becoming more common is using an Internet (or intranet) web site that invokes COM servers to generate web pages. This allows for simple rapid remote deployment of applications. The Foxisapi sample that ships with VFP5 is a sample COM server that generates web pages.

Creating COM Servers

To create a COM server (in VFP5 and VFP6), all you need to do is create a MYCLASS.PRG:

DEFINE CLASS myclass AS form OLEPUBLIC
	Datasession=2	&& private datasession
	PROCEDURE MyDoCmd(cCmd)
		&cCmd	&& just execute parm as if it were a fox command
	FUNCTION MyEval(cExpr)
		RETURN &cExpr	&& evaluate parm as if it were a fox expr
ENDDEFINE

Actually, you can get by with only 1 method, and if you want to return a value from that method, then just pass in the string "return <<cExpr>>"

For a Visual class stored in a VCX, just choose the OLE Public checkbox in the Class Info dialog. The only difference between this class and a normal VFP class is that it's been marked as being OLE PUBLIC. This is not a property of the class, (which means its not inheritable), but an indicator to VFP to publish this class to the world outside of VFP. When you subsequently BUILD an EXE or DLL (DLLs must have at least one OLE Public class), then the result will be a COM server that can be invoked from any other COM client. Now we'll build it and test it using VFP as the client of the Myclass server.

BUILD PROJECT myserver FROM myclass
BUILD EXE myserver FROM myserver	&& or DLL
*now test this COM server:
ox = CreateObject("myserver.myclass")	&& create the server object
ox.mydocmd("use home()+’samples\data\customer’") && use a table
?ox.myeval("RECCOUNT()")	&& get the record count

This is powerful stuff! Just to see how powerful, fire up Excel (this works with both VFP5 and VFP6) and choose Tools>Macros> Visual Basic Editor. (Alt-F11), Insert>Module, then paste the VBA code below. You can also use MS PowerPoint or MS Word too!

Function foo()
    Dim ox As Object
    Set ox = CreateObject("myserver.myclass")
    ox.mydocmd ("set exclusive off")
    rem be sure to use the correct path
    ox.mydocmd ("use HOME()+'samples\data\customer'")
    nrecs = ox.myeval("reccount()")
    nfields = ox.myeval("fcount()")
    For i = 1 To nrecs
        For j = 1 To nfields            
            Sheet1.Cells(i,j).Value = ox.myeval("eval(field("+ Str$(j)+ "))")            
        Next
        ox.mydocmd ("skip")
    Next
End Function

Hit the F5 button to GO and the macro executes: it creates an instance of the server and assigns it to the variable OX. Then it instructs the server to open a table, get the number of records and fields, and fills in the spreadsheet with the values from the entire table.

Obviously the COM server in this sample (with only the MyDocmd and MyEval methods) doesn't have much application specific code. It could have had a method that could have done a lot of calculation or data manipulation pertinent to the application.

Notice that the CreateObject line uses a SET command. In Visual Basic, when assigning an Object to a variable, you have to use the SET command. The only other difference between this code and similar code in VFP5 is the DIM command. In this sample, Excel is the COM client and VFP is the COM server. The code below (which can run in VFP versions since VFP3 is very similar, but it uses VFP as the COM client and Excel as the COM server. (The SCAN command and other improvements could have been used here, but then the similarity between VB and VFP code would be more obscure)

ox=CreateObject("excel.application")
ox.workbooks.add
ox.visible=1
SET EXCLUSIVE OFF
USE HOME()+'samples\data\customer'
FOR i= 1 TO RECCOUNT()
	FOR J = 1 TO FCOUNT()
		ox.activesheet.cells(i,j).value = EVAL(FIELD(j))
	NEXT
	SKIP
NEXT

COM Plumbing

COM Interfaces

Did you ever hear of the game "20 questions"? It's sometimes called Botticelli. The first person (the leader) thinks of an object (lets say the family dog), then announces "I am thinking of a Thing" The Guesser then can pose a series of 20 questions to which the Leader can respond Yes or No. The Guesser can win by guessing the object in 20 or fewer questions. "Is it a living thing?" "Yes"… "Is it a person?" "No" "Is it an Animal?" "Yes". The guesser now knows that the object in question has the properties and methods of a living thing, so it has the ILivingThing interface, does not have the IPerson interface, but does have the IAnimal interface.

The mother of all: IUnknown

All COM objects have interfaces by which they can be manipulated. Every COM interface inherits from the mother of all Interfaces: IUnknown, which has 3 methods and no properties. The QueryInterface method allows the client to interrogate the server for another Interface. The AddRef method simply increments an internally maintained reference count, which the Release method decrements. Some pseudocode for the 20 questions game:

	Ox=CreateObject("foobar.thing")
	IF ox.QueryInterface("ILivingThing" )
		IF ox.QueryInterface("IPerson")
		ELSE
			IF ox.QueryInterface("IAnimal")
		…

When we say that an interface inherits from IUnknown, it means that the first 3 methods of that interface are QueryInterface, AddRef, and Release. Because every interface inherits from IUnknown, the first 3 methods of every interface are well defined.

Let's define another interface called ILivingThing. It's first 3 methods are QI, AddRef, and Release. Lets add another method called Move and a property called Size. Each property of an interface actually is represented as 2 different methods: a property_put and a property_get method. We then have

	ILivingThing
		QueryInterface(QI params)	(from IUnknown)
		Addref				(from IUnknown)
		Release				(from IUnknown)
		Move(sDirection)		(from ILivingThing)
		Put_Size(nNewsize)		(from ILivingThing)
		Get_Size     			(from ILivingThing)

Early Binding (aka VTable Binding)

The interface can be thought of as a pointer to a table of function addresses. (In reality, it’s a pointer to a structure, the first member of which is a pointer to a table of function addresses.) The first 3 addresses are those of the IUnknown methods. This table is called the Virtual Function table or VTable. To call one of these methods, all you need to do is push the required parameters (if any) on the stack and call the function at the appropriate address in the VTable. This is easy for any C++ client (the VTable just happens to be in the identical format that C++ uses). Note that this requires that the client code have intimate knowledge of the server at compile time: the client needs to know that the Move method is the 4th method in the function table when the client code is compiled. If the server code were to change this to be the 6th method, then the client would have to be recompiled or else the client/server communication would break. The same applies to methods. If a method is modified from taking 3 parameters originally to taking 4 parameters in the new version, early binding clients would fail.

What about other languages in which it's difficult to call a function in a VTable, such as VB or VFP ?

IDispatch

IDispatch is a COM Interface by which a COM client can invoke methods or change properties without the two disadvantages mentioned above. It has 4 methods of it's own, and thus has a total of 7 methods (because All com interfaces inherit from IUnknown, which means the first 3 methods are those of IUnknown).

	IDispatch
		QueryInterface(QI params)	(from IUnknown)
		Addref				(from IUnknown)
		Release				(from IUnknown)
		GetTypeInfoCount()		(from IDispatch)
		GetTypeInfo()			(from IDispatch)
		GetIDsOfNames()			(from IDispatch)
		Invoke()			(from IDispatch)

The 2 main methods of IDispatch are GetIDsOfNames and Invoke. Suppose we want to invoke the Move method of the ILivingThing interface. Here's some pseudocode:

	IdMove = GetIDsOfNames("move")
	… error checking…
	Invoke(idMove,…)

We call the GetIDsOfNames method to get the ID of the desired method. This ID can be any identifier. Then we call the Invoke method, passing this ID. The Invoke method knows the association of the ID with the Move method and thus invokes the right method. Note that this works at run-time, and thus doesn't require the client to know anything about the server at compile time. Also, this interface works well with languages such as VB or VFP that allow the user dynamically to create servers and to invoke their methods with no compilation required. This makes IDispatch a very versatile interface for COM object manipulation.

However, (and you were wondering what's next!) there are some problems associated with IDispatch. Some methods require no parameters, while others might require several. Each of these must be callable via Invoke, and thus the parameters must be wrapped into a parameter package that can be passed to Invoke as a single parameter. This parameter package must then be unwrapped to be used by the server method. The VTable method didn't have to have this parameter mucking going on, and thus was more efficient.

Dual Interfaces

Along came dual interfaces to solve this problem. A Dual interface is one that can be invoked both by VTable binding and IDispatch. All Dual interfaces must inherit from IDispatch, which means it's first 7 member functions are the same as IDispatch (whose first 3 methods are from IUnknown). The rest of the functions are the functions of the interface as they would normally appear in the VTable.

	ILivingThing
		QueryInterface(QI params)	(from IUnknown)
		Addref				(from IUnknown)
		Release				(from IUnknown)
		GetTypeInfoCount()		(from IDispatch)
		GetTypeInfo()			(from IDispatch)
		GetIDsOfNames()			(from IDispatch)
		Invoke()			(from IDispatch)
		Move(sDirection)		(from ILivingThing)
		Put_Size(nNewsize)		(from ILivingThing)
		Get_Size			(from ILivingThing)

The methods of ILivingThing can be called either using the IDispatch interface or using the VTable of the ILivingThing interface directly. This gives Dual interfaces the best of both worlds, although it increases the size of the VTable by the 4 functions of IDispatch and requires the packaging/unpackaging of parameters at both ends.

For our first simple VFP COM server above the interface looks like:

	IMyClass
		QueryInterface(QI params)	(from IUnknown)
		Addref				(from IUnknown)
		Release				(from IUnknown)
		GetTypeInfoCount()		(from IDispatch)
		GetTypeInfo()			(from IDispatch)
		GetIDsOfNames()			(from IDispatch)
		Invoke()			(from IDispatch)
		MyDoCmd(cCmd)			(from IMyClass)
		MyEval(cExpr)			(from IMyClass)

VFP5 automation servers do not support early binding, they are only IDispatch interfaces, so the interfaces have only exactly 7 member functions: 3 from IUnknown an 4 from IDispatch. VFP6 automation servers support Dual interfaces.

When invoking a server via the VTable, the client must have intimate knowledge of the server at compile time. This knowledge must be accessible in a language independent way. If the server were to change it's interface, then the client must be recompiled.

Type Libraries

A type library is a file that can be either free-standing or embedded as a resource inside an EXE or DLL. It is a language independent method of publishing the interfaces, properties, and methods of a COM object. It can contain help strings, Help ContextIDs, parameter names and member names (of properties and methods).

The type library generated by VFP5 is not accurate, and cannot be used to make early binding clients to VFP5 servers.

The VFP6 generated Type Libraries are accurate. They contain the method and parameter names of OLE Public methods. If there's a description in the Description of the class in the VCX, then that description is put in the type library as a help string.

You can view a type library using a variety of tools. For example, use the Object Browser in Excel, the Class Browser in VFP, or the OLE Viewer in VC to view a type library. You can see that type libraries can contain entire object models of the server application, and can be rather extensive.

Be aware that when a type library is being viewed by some tool, the server cannot be rebuilt because the type library is rewritten on every server build.

The Type Library is the tool by which COM clients can determine what members an interface has. For example, try this in the Excel VBE editor. Select Tools>References and choose the Myserver TypeLibrary created by VFP6 above.

Function foo2()
    Dim oVtable As New myserver.myclass
    Dim odispatch As Object
    MsgBox (oVtable.myeval("sys(2334)")), 0, "Vtable"
    Set odispatch = oVtable
    MsgBox (odispatch.myeval("sys(2334)")), 0, "IDispatch"
    Set odispatch = Nothing
End Function

This code creates an instance of the server when the first DIM line is executed. The sys(2334) function returns 1 if the call is made via VTable, and 2 if the function was called via IDispatch. The first MsgBox calls the server using VTable binding and displays a 1, while the 2nd MsgBox uses IDispatch, and thus shows a 2.

Accurate Typelib Demo:

Start Excel VB editor and choose Tools>References> myserver type library.

Watch the tooltips as you type various parts of the following:

Dim ox as new myserver.myclass
MsgBox ox.myeval("version(1)")

C++ Demo:

How much code does it take to create a C++ program that can use the VFP Myserver COM server above? Let’s try it. Start VC, choose File>New> Projects. Type in Mytest as the name of the project, and choose Win32 Application as the project type. Choose File>New>Files>C++ Source file, Add to project, and name it myserver.cpp. Paste the following code:

#include "windows.h"
#import "c:\fox\test\myserver.tlb"	//use the full path to the server tlb

int WINAPI WinMain(HINSTANCE, HINSTANCE, LPSTR , int)
{
	using namespace myserver;	//use the myserver namespace
	ImyclassPtr pmyclass;    // declare a smart ptr to the server
	CoInitialize(0);			// initialize COM
	pmyclass.CreateInstance("myserver.myclass");	//create an instance
	_variant_t myvar, vresult;	// declare 2 variant variables.
	_bstr_t res;				// declare a bstr variable
	myvar = "version(1)";	// assign the variable
	vresult = pmyclass->myeval(&myvar);	//invoke the method
	res = vresult.bstrVal;	// get the Unicode result
	MessageBox(0,(char *)res,"",0);	//show it
	pmyclass = 0;				// release the server
	CoUninitialize();		//uninitialize COM
	return 0;
}

Hit the F5 button to go, and you see the messagebox with the version(1) result!

If you look carefully, the #import line actually does a lot of work. It takes the TLB and translates it to MyServer.TLH and MyServer.TLI files, both of which are C header files that are #included.

You’ll notice that the VB editor can actually prompt you as you type. When you type the ‘.’ after the "ox", the possible members of myclass are shown in a dropdown. Because VB is a strongly typed language, and because you specified the Type Library as a reference, the information to prompt you as you type for statement completion is available.

Demo: Java

To use your myserver COM server in a java program Run the Java Type Library Wizard (from the Devstudio Tools menu), and choose to import the Myserver type library. Choose File>New>Project and use the Java Applet Wizard and call it Jmyserver. Choose all the defaults in the Java Applet Wizard.

In the file jmyserver.java, imports section near the beginning, put
	import myserver.*;

In the class definition just a few lines below, put
	private Imyclass m_pf;

Just before the while (true) for the animation, put in the following
		com.ms.com.Variant v1 = new com.ms.com.Variant();
		com.ms.com.Variant v2 = new com.ms.com.Variant();
		String mystr;
		v1.VariantClear();
		m_pf = (Imyclass) new myclass();
		v1.putString("version(1)");	// fox expression
		v2 =m_pf.myeval(v1);
		mystr = v2.getString();

Just after the line "displayImage(m_Graphics)" in the While (true) loop, add
		m_Graphics.drawString(mystr,10,10);

Run the applet. You'll see the fox version as a string show up. 
By the way, some versions of VJ crash while exiting IE after the demo. 
This has nothing to do with the code we added.

Class Member Visibility

If you look at the myserver.TLB type library that was generated from our small sample above, you see that there are many properties and methods that are in the type library in addition to the 2 member functions that we added: MyDocmd and MyEval. AddObject, BaseClass, Class, ClassLibrary, .... are all members of the OLE Public class MyClass. These members are by default public, and thus are published in the type library for all COM clients to see. It’s not good OOP programming practice to publish all these, so it’s a good idea to make these properties Hidden or Protected. In the VCX Class Properties dialog, you can multi-select a group of properties and change their visibility.

GUIDS

When dealing with COM servers, you can’t help but notice all these weird strings like "7E5A5C95-6DE2-11D1-BEFF-0080C74BBCD5" These are called GUIDs, an acronym for Globally Unique Identifier. (Sometimes they’re called UUIDs, or Universally Unique Identifier.) The GUID is virtually guaranteed to be unique in time and space. In other words, if you generate a GUID, nobody else in the past or future, or on a different machine can generate the same GUID. GUIDs are used as IDs for various parts of COM that need to be unique.

If Widget Software Inc creates a COM Interface called IFooBar that’s used for a COM component on your machine, it might collide with an interface called IFooBar on your machine that you might create.

You can see the GUIDs that are generated when you create a VFP COM server. Look at the MYSERVER.VBR file that gets generated from the sample VFP COM server above. The VFP5 version will be a little different from the VFP6 version. This file can be used by the remote automation manager to register the server on a different machine from the one that created the server. When you build a VFP automation server, it’s automatically registered on your machine. To deploy on a different machine, you can use the VBR file or you can use the self-registration techniques described later.

Here is the VBR of the Myserver project above:

VB5SERVERINFO
VERSION=1.0.0

HKEY_CLASSES_ROOT\myserver.myclass = myclass
HKEY_CLASSES_ROOT\myserver.myclass\NotInsertable
HKEY_CLASSES_ROOT\myserver.myclass\CLSID = {149C74C0-D2F3-11D1-988E-00805FC792B1}
HKEY_CLASSES_ROOT\CLSID\{149C74C0-D2F3-11D1-988E-00805FC792B1} = myclass
HKEY_CLASSES_ROOT\CLSID\{149C74C0-D2F3-11D1-988E-00805FC792B1}\ProgId = myserver.myclass
HKEY_CLASSES_ROOT\CLSID\{149C74C0-D2F3-11D1-988E-00805FC792B1}\VersionIndependentProgId = myserver.myclass
HKEY_CLASSES_ROOT\CLSID\{149C74C0-D2F3-11D1-988E-00805FC792B1}\LocalServer32 = myserver.exe /automation
HKEY_CLASSES_ROOT\CLSID\{149C74C0-D2F3-11D1-988E-00805FC792B1}\TypeLib = {149C74C2-D2F3-11D1-988E-00805FC792B1}
HKEY_CLASSES_ROOT\CLSID\{149C74C0-D2F3-11D1-988E-00805FC792B1}\Version = 1.0
HKEY_CLASSES_ROOT\INTERFACE\{149C74C1-D2F3-11D1-988E-00805FC792B1} = myclass
HKEY_CLASSES_ROOT\INTERFACE\{149C74C1-D2F3-11D1-988E-00805FC792B1}\ProxyStubClsid = {00020424-0000-0000-C000-000000000046}
HKEY_CLASSES_ROOT\INTERFACE\{149C74C1-D2F3-11D1-988E-00805FC792B1}\ProxyStubClsid32 = {00020424-0000-0000-C000-000000000046}
HKEY_CLASSES_ROOT\INTERFACE\{149C74C1-D2F3-11D1-988E-00805FC792B1}\TypeLib = {149C74C2-D2F3-11D1-988E-00805FC792B1}
HKEY_CLASSES_ROOT\INTERFACE\{149C74C1-D2F3-11D1-988E-00805FC792B1}\TypeLib\"Version" = 1.0


; TypeLibrary registration
HKEY_CLASSES_ROOT\TypeLib\{149C74C2-D2F3-11D1-988E-00805FC792B1}
HKEY_CLASSES_ROOT\TypeLib\{149C74C2-D2F3-11D1-988E-00805FC792B1}\1.0 = myserver Type Library
HKEY_CLASSES_ROOT\TypeLib\{149C74C2-D2F3-11D1-988E-00805FC792B1}\1.0\0\win32 = myserver.tlb
HKEY_CLASSES_ROOT\TypeLib\{149C74C2-D2F3-11D1-988E-00805FC792B1}\1.0\FLAGS = 0

As you can see from the VBR file, there’s a particular GUID called the CLSID for the string "myserver.myclass" (which is referred to as the ProgID). Another GUID is used for the Interface, and one more for the Type Library.

The line that says "LocalServer32 = myserver.exe /automation" indicates the server to execute. The registry actually has the EXE name preceded by it’s full path on the install machine. That’s how COM finds the EXE server. For a DLL server, the VBR would have these lines:

HKEY_CLASSES_ROOT\CLSID\{69F21492-D2B7-11D1-BF21-AC3498000000}\InProcServer32 = myserver.dll
HKEY_CLASSES_ROOT\CLSID\{69F21492-D2B7-11D1-BF21-AC3498000000}\InProcServer32\"ThreadingModel" = Apartment

The 2nd line here is the only indication to COM that VFP6 servers are Apartment Model threaded. See the Threading section for more details.

How COM Instantiates an Object

When a COM client creates an instance of a COM server, internally it executes the OLE function CoCreateInstance(). This function will search the registry for the desired server and use that information to launch the server. Given a ProgID such as "Myserver.Myclass", the first thing to do is convert that to a GUID called the CLSID. Then looking for the server to create is just a matter of looking up the registry entry for \HKEY_CLASSES_ROOT\CLSID\{guid} to find the InProcServer32 key for DLL servers or the LocalServer32 key for EXE servers.

COM Server Deployment Issues

Self-registration

DLL servers that are self registering can be registered via

	Regsvr32 myserver.dll 		to register a server
	Regsvr32 /u myserver.dll 	to unregister a server

Or, you can use this VFP code to register or unregister a server.

declare integer DllRegisterServer IN myserver.dll
declare integer DllUnregisterServer IN myserver.dll
?DllUnregisterServer()
?DllRegisterServer()

Registering a server means the registry entries required for the server will be added to the machine registry. By the way, OCXs are DLL COM servers too, and they can be registered in the same way. OCXs are a special kind of COM server that support a well-defined set of interfaces. All VFP5 and VFP6 servers are capable of self-registration.

EXE servers are self-registered like so:

	Myserver /regserver
	Myserver /unregserver

Different types of COM servers

DLL and EXE servers

There are 2 main different types of COM servers: DLL (sometimes referred to as In-Process) and EXE (sometimes referred to as Local) servers. DLL servers must exist in the same process as their host. EXE servers exist as separate processes. EXE servers require interprocess communication between client and server, which can be relatively slower than the direct communication offered by a DLL server. When a COM client calls a DLL server method, that call can be just as fast as if the COM client were calling a routine of it's own. For an EXE server, the method call must be translated (marshalled) from the process space of the client to that of the server. Because the DLL server must live in the same process space as the host, a crashed DLL server can easily crash the client. EXE servers are more robust: a crashed EXE server may not crash the client.

Actually, a DLL server doesn't have to live in the process space of the client: If you use Microsoft Transaction Server (code-named Viper), then the DLL will live in a process created by MTS, sometimes called Viperspace.

An EXE server (and a DLL server living in MTS) can be "remoted"… it can be put on a different machine. Thus you can create a server to create the timely end of month reports and let it execute on your boss's powerful machine, freeing your machine to play games.

Single-Use, Multi-Use and not Creatable

Furthermore, OLE Public classes in VFP can be marked as Single-Use, Multi-Use and not creatable. This can be changed in the 3rd tab of the Project Info dialog.

Single use means that only a single instance of the class can be created per client. Do not confuse this with Single User. If another instance is requested of a single-instance server,

	ox = CreateObject("myserver.myclass")
	oy = CreateObject("myserver.myclass")

then for an EXE server, an entire new EXE is launched. For a VFP6 DLL server, only one instance will be created, and the 2nd request will result in an error message. Even executing the same command twice in a row for a VFP6 single-use DLL server

	ox = CreateObject("myserver.myclass")
	ox = CreateObject("myserver.myclass")

will result in an error. The reason for this is that the VFP assignment statement evaluates the right hand side first before releasing the original contents of the left hand side variable. That means the 2nd instance will attempt to be created first before the first one is released.

Multi-use servers means multiple instances of the OLE Public class can be instantiated from the same server. If you rebuild the sample server to be a Multi-Use EXE or DLL server:

ox=CreateObject("myserver.myclass")		&& Create a first instance
oy=CreateObject("myserver.myclass")		&& Create a 2nd instance
?ox.mydocmd("public zz")			&& Create a public var in 1st 
?ox.mydocmd("zz='set from ox'")			&& give it a value
?oy.myeval("zz")				&& Query it from 2nd

The 2nd instance can see the public variable created by the first instance. More interesting scenarios can be made with 2 separate client instances, rather than 2 instances from the same client. For DLL servers, each separate client instance (which lives in it’s own separate address space) would get it’s own private copy of the DLL server. For EXE servers, you can launch 2 instances of VFP to be the 2 separate clients. In one instance, Create a first instance,

ox=CreateObject("myserver.myclass")		&& Create a first instance
?ox.mydocmd("public zz")			&& Create a public var in 1st
?ox.mydocmd("zz='set from ox'")			&& give it a value

* and in the 2nd instance:
oy=CreateObject("myserver.myclass")		&& Create a 2nd instance
?oy.myeval("zz")				&& Query the value from the 1st instance

In this case, there are 2 clients, but only one instance of the server EXE is instantiated, which serves up 2 instances of the OLE Public class. You can use the Windows NT Task Manager or some other tool to see that there is indeed only one instance of MYSERVER.EXE running.

We have proven that the 2 instances of the class are in the same EXE by creating the public variable ZZ in one that is visible in the other. To prove that the 2 instances of the class are indeed different instances, we can use the new AddProperty method in VFP6 to add a property to the first instance and see if that property is visible in the 2nd instance:

ox.AddProperty("foo","default value")
?ox.MyEval("this.foo")
*and in the 2nd instance,
?oy.Myeval("this.foo")	&& causes an error

Not creatable is useful if you have a Visual Class Library with several OLE Public classes in it and this VCX is shared amongst multiple projects. You may not want to have all the OLE Public classes in the VCX made public for each project, so you can select the ones you’d like to be not creatable.

COM Server Programming Issues

UI-less servers

VFP6 DLL servers cannot do any kind of User Interface operation. VFP5 allowed a window to be shown, but it couldn’t really interact with the user. The reason for this is that the main purpose of the server is to perform a calculation or do data manipulation. Doing UI is not the purpose of a server. For example, a COM server that generates web pages would be called from a Web server. There would probably not be a user on that machine to do user interface operations.

In fact, Internet Information Server (IIS) on Windows NT runs in the security context of a service, which means that it doesn’t even have a desktop. If a COM server called from IIS were to call the MessageBox function, the server would hang in an infinite loop, because the code think it’s displaying a MessageBox, but no actual MessageBox would appear. The service doesn’t have a desktop, and so all UI functions do not even appear on the user’s desktop.

Visual Foxpro 6 has a new feature in COM servers that is always present in DLL servers, and can be turned on in EXE servers. SYS(2335) will turn off User Interface operations. If a user interface operation (such as READ EVENTS, MessageBox, Wait Window, etc.) were to be executed, it would cause an error event to be executed.

EXE servers, on the other hand, can use User interface operations. An EXE server has it’s own process space and thus is allocated processor time to execute, and thus can handle events. A DLL server is hosted in the client’s process space and is given execution time only when the client calls into the server. There is no processor time given to the server when the server is not executing a method, so the DLL server cannot process events.

Error handling

Servers that try to show UI is a very big reason why Error handling is so important in COM servers, and particularly so in DLL servers. If the client were to invoke a method on the server that caused some sort of error (such as "File Not Found" or "Access Denied"), it wouldn’t be good to have the server just throw up a messagebox indicating the error. The developer should use the Error Method of the OLE Public class to handle such errors gracefully. A new function in VFP6 called COMReturnError will actually cause a COM Error object to be created and returned to the COM client. It takes 2 parameters: the Source and the Description. You can put any strings you want into these parameters. This example method can be pasted right into the Myserver sample above.

	FUNCTION Error(nError, cMethod, nLine)
		COMreturnerror(cMethod+'  err#='+str(nError,5)+'  
line='+str(nline,6)+' '+message(),_VFP.ServerName)
		&& this line is never executed

You can invoke this error method by calling the MyDocmd method with an invalid command:

ox = CreateObject("myserver.myclass")	&& create the server object
?ox.mydocmd("illegal command")	&& causes an Error to occur

The error that occurs in the server is trapped by the MyClass::Error method which then causes the server to abort processing and return the COM Error object with the Source and the Description filled out.

?aerror(myarray)
list memo like myarray
MYARRAY     Pub    A  
   (   1,   1)     N  1429        (      1429.00000000)
   (   1,   2)     C  "OLE IDispatch exception code 0 from mydocmd  
                      err#=   16  line=     2 Unrecognized command v
                      erb.: c:\fox\test\myserver.exe.."
   (   1,   3)     C  "c:\fox\test\myserver.exe"
   (   1,   4)     C  "mydocmd  err#=   16  line=     2 Unrecognized
                       command verb."
   (   1,   5)     C  ""
   (   1,   6)     N  0           (         0.00000000)
   (   1,   7)     N  0           (         0.00000000)

Some helpful _VFP Properties

A few properties are useful for COM servers, some of which are new to VFP6. First, what is this _VFP object ? If another application (such as Excel or VC) were to do CreateObject("VisualFoxpro.Application") then the object returned to that client would be the _VFP object. Thus, the _VFP object is the same object that would be returned to an external client.

The _VFP.Servername property is the fully qualified path and name of the server. In the example above, it’s "c:\fox\test\myserver.exe"

The _VFP.FullName property gives the fully qualified path and name for the fox runtime that’s used for the server.

The _VFP.Startmode property returns an integer. 0 indicates that VFP was started normally as an interactive desktop session. 1 means that VFP was started as an EXE automation server (a client did CreateObject("VisualFoxpro.Application")). 2 means it’s an EXE COM server, and 3 means it’s a DLL COM server.

Processes & Threads

What’s a Thread ? What’s a Process ? A Process is something about which you’re quite familiar. If you launch Microsoft Word and VFP, then you have two separate processes running, even though your computer probably has only one CPU. The Operating System actually allocates execution time between processes and switches from one to the other.

In a non-preemptive multitasking system (a task is synonymous with a process) such as Windows 3.1, a process would be given execution time and it would be up to that process to yield execution to another process. If the process didn’t yield, then it wasn’t a good player and would keep the user from switching to another application. In a pre-emptive system, the operating system automatically switches from process to process, regardless of whether that process yields the processor.

The switching of the CPU from one task to another is quite expensive, in terms of time and resources. It takes lots of memory and register saving and restoring to do such a context switch. The entire address space of one application is switched out for another’s.

This architecture doesn’t work too well for an application that is a web server. A user on a browser somewhere in Hawaii hits your web site and your web server starts processing the web request. Along comes another hit from another user in Alaska, and it has to wait until the first one is completed. A brute force solution to this problem is to just have multiple server processes serving these requests, so that they can be performed more or less simultaneously. This is the approach taken by CGI architecture web sites. (Each CGI web hit would spawn a new process that creates a web page, then terminates.) However, this is resource intensive and inefficient.

A much better solution is to have multiple threads within a process able to process web requests. Every process has exactly one main thread, but it can also create multiple private threads for its own use. The operating system not only is perfectly happy doing the context switching as before for processes on the machine, but it is also doing the context switching and execution time allocation for the private threads.

The cost of a context switch for threads is much smaller than that for processes. The virtual address space for the threads is the same.. only the registers need to be switched. Thus threads gain a lot of efficiency. IIS and the ISAPI extension architecture (see the FOXISAPI sample) is based on multiple threads. Each web hit uses one of a pool of existing threads to service the request.

Programming for multiple threads

Writing code for multiple threads is different from writing single threaded code. Imagine that while one of your procedures is executing, the processor could swap you out and swap in another thread which wants to execute the same procedure. This swapping could occur at any time, which makes it very difficult to test.

Consider the following pseudo code:

	FUNCTION MyProc
*		LOCAL myvar
		Myvar = 44
		If myvar = 44
			Myvar = 55
		Endif
		Return myvar

With the LOCAL statement commented out, what will the return value of this function be in a multithreaded environment ? It looks like the 55 should be returned. However, it’s possible that after the assignment of the value 44, another thread could execute, change the value of myvar to 55, then your thread would resume at the IF statement with myvar = 55. One solution to fix this is to use the LOCAL statement. Another solution would be to lock out all other threads using some sort of thread synchronization techniques.

Process switches are relatively safe

Even though a process switch can occur after any line of assembly code (keep in mind that all code executed by the processor is assembly code), programmers don’t have to worry too much about being swapped out. The entire processor state is saved and restored, along with all global variables, handles, windows, stacks and CPU registers. The registers contain the stack pointer and the instruction pointer. The swap includes the entire logical 4 gigs of address spaces. Every processes’ logical address space is protected from all other processes this way. In reality, a much more efficient memory re-mapping is done, rather than moving all the memory around.

Thus, a programmer doesn’t have to do any defensive programming around a processor switch. Potential process switch problems can still occur. Imagine, for example, starting a word processing program like MSWord and opening the file FOO.DOC. Start up Windows Explorer (which is another process) and delete the file. Switch back to Word, and an error can occur because the file no longer exists.

Threads

Threads are mini-tasks within a process. Every process has at least one thread. The OS allocates execution time between each thread of every process running on the system. Each thread has its own stack. A thread switch is a much faster, more efficient swapping. Just the machine registers are switched. This includes the stack pointer and instruction pointer. The memory space for each thread is shared, unlike a process switch. Potential problems abound.

Thread switches are unsafe

Imagine some code:

SEEK mycustomer

<thread switch>

REPLACE balance WITH balance + 10

OR

IF !FILE("customer.dbf")

CREATE TABLE customer (name c(10))

FOR i = 1 TO 1000

<thread switch>

INSERT INTO customer VALUES...

...

The thread switch could cause the record pointer to be at a different location when the values are written to the table, causing corrupted data.

To prevent such problems, some sort of thread synchronization primitives are needed.

Thread Synchronization

Critical sections and Mutexes are common means of thread synchronization. A Critical section is a lightweight shared memory object that can allow only one thread to proceed. It only works within a process boundary. A mutex can be used across the process boundary

The fixed code from above would be wrapped by critical section entry and exit code:

EnterCriticalSection

SEEK mycustomer

REPLACE balance WITH balance + 10

LeaveCriticalSection

Luckily, if you use VFP private data sessions, then each table used in each thread would have it’s own record pointer, and this level of thread synchronization is unnecessary.

In fact, VFP6+ has a new baseclass called SESSION which is very lightweight, and contains the bare minimum properties plus the "DataSession" and "DataSessionID" properties, and allows lightweight private data session objects.

VFP6 Multi Threaded DLLs protect your program in many ways so that the programming model you need to follow is the same as if each instance of an object were in a different process. For example, things like SET EXCLUSIVE ON would not allow multiple threads to share a table.

Suppose you wanted to create a LOG file for each web hit using your com server from above. Your fox program might look like:

#define OUTFILE "d:\t.txt"
Set safety off
ss=""
SYS(2336,1) 	&&Enter Critical Section (incrments count)
if NOT file ("c:\counter.dbf")
	create table c:\counter (count i)
	insert into counter values (0)
	ss  = ss + "Created"
	use
else
	ss  = ss + "not Created"
endif
SYS(2336,2)		&& Leave Critical Section (Decrements count)
*
* other code...
*
SYS(2336,1)
n = strtofile(CHR(13)+chr(10) + TRANSFORM(datetime())+' 
'+TRANSFORM(_vfp.ThreadId) ,OUTFILE,.t.)
SYS(2336,3)		&& Leave ALL critical sections (recursive)

for i =1 to 100
	SYS(2336,1)
	n = strtofile(transform(_vfp.ThreadId),OUTFILE,.t.)
	SYS(2336,2)
	if n = 0
		ss = "error"
		strtofile(CHR(13) +chr(10)+ 'error 
'+TRANSFORM(_vfp.ThreadId) ,"d:\t.txt",.t.)  
	endif
	endfor
return PROGRAM() +' '+ss

and your ASP page might look like:

<HTML>
<HEAD>
<META NAME="GENERATOR" Content="Microsoft Developer Studio">
<META HTTP-EQUIV="Content-Type" content="text/html; charset=iso-8859-1">
<TITLE>Document Title</TITLE>
</HEAD>
<BODY>

<%
    set ox = Server.CreateObject("myserver.myclass ")
	ox.mydocmd("set excl off")
	ox.mydocmd("set path to c:\fox\test")
	ox.mydocmd("clea prog")
	response.write "<br>"
	response.write ox.myeval("testprog()")
	response.write "<br>"
	response.write ox.myeval("'thread id = '+trans(_vfp.threadid)")
	ox.mydocmd("clea prog")

	ox.mydocmd("use c:\counter shared")
	ox.mydocmd("replace count with count+1")
	response.write  ox.myeval("'<br><font size=5 color = red>Count = '+trans(counter.count)+'</font>'")
	response.write "<br>"
	ox.mydocmd("use")


%>

<br>
this is a test page
let's see if this works

</BODY>
</HTML>

The Testprog above first tests to see if COUNTER.DBF exists. If it doesn’t exist, it’s created. This is problematic code in a multithreaded scenario: One thread can come in, find the table doesn’t exist, and starts creating it, and another thread comes in and does the same thing. To avoid this kind of collision, the code is wrapped with calls to Enter and Leave a critical section. This allows only 1 thread at a time to execute the guarded code. If thread A is execuging the guarded code, and thread B tries to enter the critical section, then thread B is put into an efficient wait state until the critical section is available. Thus it’s important to keep the guarded code to a minimum to avoid making other threads wait.

The next code in Testprog tries to write to a file using STRTOFILE(). STRTOFILE will return 0 if it fails. If you have multiple threads hitting the server, the STRTOFILE can fail if the file is being written with one thread, and the other thread tries to write to it. Without the critical section, a collision can occur fairly often, and "error" is returned.

SYS(2336,1) enters the critical section. It just increments a count. SYS(2336,2) leaves the critical section by decrementing the count. When the count goes to 0, then the critical section can be entered by other threads. Thus it’s important to make sure these calls are paired. For example, guarded code might call a subroutine, which itself might have guarded code.

SYS(2336,3) will leave a critical section enough times to make the count 0. This will ensure the critical section is freed. This is useful, for example, in error handling code.

The ASP code above opens the table COUNTER.DBF and increments the count. This code is threadsafe because the REPLACE command will lock a record before doing the replace and unlock it after. Thus another thread coming in will have to wait for the record to be unlocked before doing the REPLACE.

Note: the SYS(2336) enter critical section is only available in multithreaded DLLs created by VFP6.0 SP3

However, consider code like this

SEEK Customer
INSERT INTO customer (balance) values (balance + amount)

This code is NOT threadsafe because another thread could do a SEEK to a different customer. To make this threadsafe, you can wrap it with a critical section, but a better solution would be to use a private datasession. Using a FORM or SESSION baseclass with Datasession=2 ensures this. Private datasessions in this scenario are logically the same as 2 different instances of the same form in the same instance of VFP.

Web development tips

When developing DLL COM servers for ASP and IIS, every code change to the DLL requires IIS shutdown and restart because IIS will cache the server instance, and the DLL will be in use.

From a command window, you can type "kill –f inetinfo" to kill the IIS process, which will release the DLL. To restart, you can type "net start w3svc".

There has been a registry key for IIS to not cache the DLL over the various versions of IIS, but a simpler method is to not put the code into the server at all.

As seen with the TESTPROG.PRG and ASP above, you can change the code that the COM server runs just by Compiling the PRG. The ASP just sets the path to the directory where the PRG is, and executes the program using the myeval method of the server. After the method is run, calling mydocmd with "Clear Program" allows the TESTPROG.FXP compiled code to be released, and the program can be modified again. You can also comment out the line that calls the PRG if it’s still not released (in case of a script error, for example), then hit the page again with a browser.

Apartment Model Threading

What happens when one thread creates an instance of a COM server and another thread needs to call a method on that server?

Single threaded servers are the simplest COM threading model. This is the only model available using Windows NT 3.5. and the only one that VF5 supports (VFP5 was released in August of 1996). On NT 3.5, multiple threads in a single process were not allowed to use COM. A COM server just used the same thread as the client. The client cannot call into the COM server using a different thread. This made it impossible to create multi-threaded COM servers or clients.

With NT3.51 and Win 95, multithreaded COM servers and clients are possible. With Apartment model (sometimes called Single Apartment Model) threaded COM objects, all threads in a process can use COM and all calls between client and server are synchronized by COM. The client thread that created the COM server is called the "owning" thread of that object. When that thread calls into the server object, the calls are made directly. If another thread wants to call into the object, the call is marshalled from that thread to the owning thread.

The Free-threaded or Multi-Threaded Apartment Model does not synchronize COM communication, and COM objects must synchronize themselves to protect from simultaneous access. Interfaces are not marshalled between threads. This threading model is available only in Win NT4 and Win 95 with DCOM 95.

Multi-Processor Machine

With a single processor machine, only 1 thread can run at a time and thread switches occur in a time-sliced fashion to allow the illusion of multiple simultaneous threads. With an n processor machine, with n > 1, n threads can run simultaneously. This means additional pitfalls can occur to the programmer. For example , the simple assembly instruction

inc g_nCount

just increments a global variable. On a single processor machine, this will be thread safe (with caveats), but on a multiprocessor machine, this is not. Why is it not safe ? Aren’t thread switches done after the completion of an ASM instruction? Yes they are, but it’s not thread safe because one thread could read the value, the other thread (which is running on another processor, sharing the same memory) could change the value back, and the first thread puts in what it thought was g_nCount + 1.

The easy fix is to use the InterlockedIncrement instruction, which essentially causes this code to execute:

lock inc g_nCount

The lock prefix causes the processor to lock that memory location from any other processor for the duration of the instruction.

MT Testing

Developing and testing a multi-threaded application requires a multi-processor machine to even see some of the possible errors that can occur. Some kinds of collisions are extremely difficult to reproduce, because they depend on the relative timing of threads. Other possible problems occur only on multi-processor machines.

MT Performance

On a single processor machine, having a multiple threaded DLL will NOT increase the throughput. In fact, the real amount of work accomplished can actually decrease because of thread switching. Suppose Clients A & B start a 1 second request simultaneously. Both threads will appear to be executing simultaneously, but neither client will be done until a little more than 2 seconds has elapsed.

The real performance gains from having a multi-threaded component can be seen when using a multi-processor machine. This way, multiple simultaneous threads can be servicing client requests. Client A & B requests will be done in a little over 1 second total.

Demo: Multi Threaded C++ client

This demo creates 10 threads and creates an instance of Myserver.Myclass on each thread. Each thread then calls a method on the server to get the thread ID and outputs that value with the Thread ID of the client to the VC Debug Output window. When you run the program, you see that the thread id of the client is the same as that of the server, and you see that the thread processing is interlaced with other thread processing. If you put the COM server into Microsoft Transaction Server, the thread ID of the client will be different from that of the server.

Start VC, choose File>New> Projects. Type in MT as the name of the project, and choose Win32 Application as the project type. Choose File>New>Files>C++ Source file, Add to project, and name it mt.cpp. Make sure the myserver project above has been built with into a DLL with the Multi-Use attribute. Paste the following code:

#include "windows.h"
#import "c:\fox\test\myserver.tlb"

#define NUMTHREADS 10
#define NUMITER 10

class CThreadObj  {
public:
	static unsigned long __stdcall RealThreadStart(void *p);
	int VirtualThreadStart();
	DWORD	m_threadId;	//and the thread ID
	int m_nthread;
};


//These 2 macros make it easier to call methods on VFP servers.
#define XDOCMD(cmd) v1 = cmd;v2 = pmyclass->mydocmd(&v1);
#define XDOMETH(meth, cmd) v1 = cmd; v2 = pmyclass->##meth(&v1)

int CThreadObj::VirtualThreadStart() {
	char szBuf[1000];
	int i;
	CoInitialize(0);
	using namespace myserver;
	ImyclassPtr pmyclass;	// declare a smart ptr to the server
	pmyclass.CreateInstance("myserver.myclass");
	_variant_t v1,v2;
	long vfpthreadid;
	XDOCMD("declare integer GetCurrentThreadId in win32api");

	for (i = 0 ; i < NUMITER ; i++) {
		//ask the server for the thread ID
		XDOMETH(myeval,"GetCurrentThreadId()");	
		vfpthreadid = v2.lVal;
		wsprintf(szBuf,
			"i = %d Thread # %d ID = %d VFPID = %d\n",
			i,m_nthread,m_threadId, vfpthreadid);
		//show the output in the VC Output Window
		OutputDebugString(szBuf);
	}
	pmyclass = 0;				// release the server
	CoUninitialize();

	return 0;
}

//this is the static real thread starting proc
unsigned long __stdcall CThreadObj::RealThreadStart(void *p) {
	CThreadObj *pobj;
	//get a pointer to the thread object
	pobj = (CThreadObj *) p;
	// and call it's thread start routine
	pobj->VirtualThreadStart();
	return 0;
}

int WINAPI WinMain(HINSTANCE , HINSTANCE , LPSTR , int ) {
	CThreadObj threads[NUMTHREADS];	//decl array of CThreadObj objects
	HANDLE harray[NUMTHREADS];		//array of handles
	for (int i = 0 ; i < NUMTHREADS ; i++) {
		threads[i].m_nthread = i;	//record the thread #
		harray[i] =		//record the thread ID
			CreateThread(NULL,0, 
				CThreadObj::RealThreadStart, //thd strt addr
				(void *)&threads[i], //parm to pass to thrd
				NULL, 
				&threads[i].m_threadId);
	}
	//now wait for all the threads to finish
	WaitForMultipleObjects(NUMTHREADS, harray, TRUE, INFINITE);
	return 0;

}

Summary

You’ve now learned how to create VFP COM servers that you can deploy to do various tasks. You can harness VFP’s legendary data processing speed and flexibility and package it into your own software components, which are now capable of high throughput. You’ve learned about COM and interfaces and how software components can be glued together for reuse and easier maintainability.