Adventures in Haskell: Dynamic loading and compiling of modules

Tags:

In my latest attempt at using Haskell, I was working on creating a kind of a plugin architecture for my application. I wanted to be able to load haskell code dynamically, without having to restart the main application.

This is doable using the GHC compiler API’s. However, the documentation for it is quite lacking in examples, and while I was able to find an article talking about it to get started, it wasn’t entirely smooth sailing even after that…

Getting started

I have adapted this code mostly from this blogpost.

Executing dynamic code is done inside a Ghc monad which represents a GHC session. As far as I can tell, you must load modules separately per each session, so I’ve split the code into two functions to make it easier to create code using dynamically loaded modules.

First, loadSourceGhc for loading Haskell source files into the GHC session

loadSourceGhc :: String -> Ghc (Maybe String)
loadSourceGhc path = let
        throwingLogger (Just e) = throw e
        throwingLogger _ = return ()
    in do
        dflags <- getSessionDynFlags
        setSessionDynFlags (dflags{
            ghcLink = LinkInMemory,
            hscTarget = HscInterpreted,
            packageFlags = [ExposePackage "ghc"]
            })
        target <- guessTarget path Nothing
        addTarget target
        r <- loadWithLogger throwingLogger LoadAllTargets
        case r of
            Failed    -> return $ Just "Generic module load error"
            Succeeded -> return Nothing
 
        `gcatch` \(e :: SourceError) -> let
                errors e = concat $ map show (bagToList $ srcErrorMessages e)
            in
                return $ Just (errors e)

This code should be relatively self-explanatory if you read the original post I’ve based this on. However, the minor changes I’ve done here are as follows:

  • The code sets SessionDynFlags with LinkInMemory, HscInterpreted and ExposePackage “ghc”.

    The first two are so that you can also reload modules – Without them, I was not able to first load module Foo, and then at a later point during the same program’s execution, reload it if the file had changes. LinkInMemory and HscInterpreted fixed this issue.

    Lastly, by using ExposePackage, the dynamically loaded code has access to the GHC API functionality, such as the functions used in this code.

  • The code uses an error logger which just throws them. This is so that we can catch more of the errors in the gcatch block. I wasn’t able to find out a way to catch all of them without using this approach.
  • The function catches any errors that occur during loading and return a Just with an error string

The second function is for executing functions inside loaded code and getting their return values:

execFnGhc :: String -> String -> Ghc a
execFnGhc modname fn = do
        mod <- findModule (mkModuleName modname) Nothing
        setContext [] [mod]
        value <- compileExpr (modname ++ "." ++ fn)
 
        let value' = (unsafeCoerce value) :: a
        return value'

This function simply takes the module’s name and a function’s name to execute. As far as I’m aware of, unsafeCoerce is the only way to get a real value out from the result.

Be careful – if you coerce a returned value into something that it isn’t, you will get unexpected results.

Using the functions

Both of the above functions must be ran inside the Ghc monad, so we need to use runGhc…

Here’s an example function which loads a module and executes a function which is expected to return a Bool

example :: IO (Bool)
example = runGhc (Just libdir) $ do
    loadrst <- loadSourceGhc "Example.hs"
    case loadrst of
        Just err -> error err
        Nothing  -> do
            b <- execFnGhc n "giveMeBool"
            return b

In closing

I’m by no means a Haskell-pro, so the above code may not be perfect. Feel free to suggest improvements :)

The full module for the functions, including all the necessary imports, can be found in this Gist. The runGhc function also exists in the GHC module.

The GHC.Paths module can be installed via cabal using the name “ghc-paths”

When compiling code which uses the GHC API, add -package ghc to the parameters:

ghc -package ghc --make foo

When using GHC API in ghci, you need to use…

:set -package ghc

If you get an error along the lines of “Could not find module `SomeModule’: It is a member of the hidden package `ghc-n.nn.n'” when using this code, it’s most likely caused by not remembering to use the above package flags.