As I mentioned in a previous post, Copernic Desktop Search 1.5 beta is currently available to the public. One of the most important new features was for me the introduction of an extensibility API.

When I wrote that first announcement, I hadn't had a close look at the API. By now I've found out that it's about extracting data from new file types, nothing more or less than that. I'd wish, and maybe I should add that to my list of things I'd like to see in CDS, they would extend that extensibility support to other areas, like creating plugins for file type preview, or even introducing completely new ranges of searchable objects. Well, maybe that's in the future.

For now, I've taken the plunge and tried to implement a custom extractor for CDS. And while I was at it, I wanted to do it in in C#. I succeeded and these are the results, maybe somebody will find this useful to implement a custom extractor that really does something worthwhile :-)

The interface

The first task was to create a C# interface from the IDL description that's given in the API documentation. My experience with COM and ActiveX stems from work I've been doing in C++ and Delphi some years back, I've been doing close to nothing with .NET InterOp, so I'm no expert at this. Nevertheless, I managed to create the following definition:

[ComVisible(true), ComImport, Guid("7E337435-5E47-40A0-B8A9-315BDD1BAE0D"),
  InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public interface ICopernicDesktopSearchFileExtractor {
  [DispId(1)] 
  void LoadURI([MarshalAs(UnmanagedType.BStr)] string uri);
  [DispId(5)] 
  void GetContentStream([MarshalAs(UnmanagedType.IUnknown)] out object contentStream);
  [DispId(9)] 
  void get_IsContentUnicode([MarshalAs(UnmanagedType.VariantBool)] out bool value);
}

The implementation

The next job was to create an implementation of the interface in a COM server. BTW, the assembly you use for this must have the Register for COM Interop flag set. This is my implementation:

[ClassInterface(ClassInterfaceType.None), Guid("1bb4b2a5-d516-4a00-868b-cfc49a84881a")]
public class CustomExtractor: ICopernicDesktopSearchFileExtractor {
  void ICopernicDesktopSearchFileExtractor.LoadURI(string uri) {
    currentURI = uri;
  }

  private string currentURI;

  void ICopernicDesktopSearchFileExtractor.GetContentStream(out object contentStream) {
    contentStream = null;
    if (currentURI != null && File.Exists(currentURI)) 
      contentStream = new IStreamWrapper(File.OpenRead(currentURI));
  }

  void ICopernicDesktopSearchFileExtractor.get_IsContentUnicode(out bool value) {
    value = false; // return true if the file contains Unicode content
  }

  const string regPath = @"SOFTWARE\Copernic\DesktopSearch\CustomExtractors";

  [ComRegisterFunction]
  public static void Register(Type t) { 
    RegistryKey rkey = Registry.LocalMachine.CreateSubKey(regPath);
    rkey.SetValue(".testfile", "{1bb4b2a5-d516-4a00-868b-cfc49a84881a}", 
      RegistryValueKind.String);
    rkey.Flush( );
  }

  [ComUnregisterFunction]
  public static void Unregister(Type t) {
    RegistryKey rkey = Registry.LocalMachine.CreateSubKey(regPath);
    rkey.DeleteValue(".testfile", false);
    rkey.Flush( );
  }
}

This implementation registers itself for a ".testfile" filetype. In reality, I simply created a few text files and renamed them to that extension.

The stream wrapper

As you can see, there's a class in use called IStreamWrapper. This is another class I had to write because CDS wants to access the extracted data from the file using the COM interface IStream. The .NET System.IO.Stream doesn't implement that interface, so I needed a wrapper class for a .NET stream that I could expose via COM. Luckily, I found that the implementation was rather straightforward and that CDS only calls two methods from the IStream interface during a normal run, so I didn't need to implement many of the methods. Here's what I came up with:

public class IStreamWrapper : IStream {
  public IStreamWrapper(Stream stream) {
    if (stream == null)
      throw new ArgumentNullException("stream", "Can't wrap null stream.");
    this.stream = stream;
  }

  Stream stream;

  public void Clone(out System.Runtime.InteropServices.ComTypes.IStream ppstm) { 
    ppstm = null;
  }

  public void Commit(int grfCommitFlags) { }

  public void CopyTo(System.Runtime.InteropServices.ComTypes.IStream pstm, 
    long cb, System.IntPtr pcbRead, System.IntPtr pcbWritten) { }

  public void LockRegion(long libOffset, long cb, int dwLockType) { }

  public void Read(byte[] pv, int cb, System.IntPtr pcbRead) {
    Marshal.WriteInt64(pcbRead, (Int64) stream.Read(pv, 0, cb));
  }

  public void Revert( ) { }

  public void Seek(long dlibMove, int dwOrigin, System.IntPtr plibNewPosition) {
    Marshal.WriteInt64(plibNewPosition, stream.Seek(dlibMove, (SeekOrigin) dwOrigin));
  }

  public void SetSize(long libNewSize) { }

  public void Stat(out System.Runtime.InteropServices.ComTypes.STATSTG pstatstg, int grfStatFlag) {
    pstatstg = new System.Runtime.InteropServices.ComTypes.STATSTG( );
  }

  public void UnlockRegion(long libOffset, long cb, int dwLockType) { }

  public void Write(byte[] pv, int cb, System.IntPtr pcbWritten) { }
}

The only methods that CDS really uses are Seek and Read.

The test setup

To try things out, I shut down CDS and moved my old index files away. Then I changed the settings for folders indexed to include only the one folder where I had created the test files with the .testfile extension. I compiled and registered the library with the classes above (for registration of a .NET assembly you use regasm, not regsvr32 as the documentation describes) and startup CDS again. I found the most reliable way to get CDS to reconstruct its index was to use the link on the Advanced page of the options dialog, that says "Clear index contents and reindex all files and folders". The suggestion from the documentation didn't work reliably for me, to use the Options/Update index/Files menu entry.

Once everything was in place, I found the whole mechanism working flawlessly. CDS would index my files and I could find them when entering search words that I knew were in the files. The only fly in the ointment, as I already hinted above, is that CDS doesn't provide any kind of preview for custom file types, and there's no (public?) way to extend it in that regard. Apart from that, thanks to Copernic for listening to those of your users who suggested an extensibility feature!