IMAP's `fetch` results with extreme memory usage and computer freezing.

[yuvallanger]

I have written the following function using the HaskellNet and HaskellNet-SSL libraries.
Using the "fetch" function results with memory usage blowing up and extreme swapping.
What am I doing wrong? Might it be a bug?

import qualified Data.ByteString.Char8       as SB8
import           Network.HaskellNet.IMAP
import           Network.HaskellNet.IMAP.SSL

main = retrieveCSV

retrieveCSV = do
    con <- connectIMAPSSL imapServer
    login con username password
    select con mailbox
    lastMsg <- last <$> search con [ALLs]
    -- this works:
    _ <- fetchHeader con lastMsg
    -- this works:
    _ <- fetchSize con lastMsg
    -- this blows up in memory usage:
    _ <- fetch con lastMsg
    undefined
    where
    imapServer = "imap.gmail.com"
    username   = "example" -- imagine this username is real
    password   = "example" -- imagine this password is real
    mailbox    = "example" -- imagine this mailbox is real

[lemol]

Hi @yuvallanger , sorry for the answer delay. Please can you give more information about the 'lastMsg' mail content? What is it actual size? Does it have any attachment? Or, the problem is for all mail message?
I tried but cannot reproduce the problem.

[mkawalec]

@lemol, @jtdaugherty This performance issue is probably related to the fact that even the ByteString implementations use a String fetcher. Would you be willing to accept PRs that fix that?

[lemol]

@mkawalec , thank you for your interest. Yes, a PR will be well come!

[mkawalec]

@lemol: Decided to go the long way and created my own imap library, https://github.com/mkawalec/imap.

[lemol]

Nice, @mkawalec !

[webcpu]

I have profiled it and found that function eval/fetchByString/ consumed several gigabytes ram.
Parse is a recursive function and very slow, and tail-recruisve optimisation doesn't help in this case.

eval :: (RespDerivs -> Result RespDerivs r) -> String -> ByteString -> r
eval pMain tag s = case pMain (parse tag (Pos tag 1 1) s) of
                     Parsed v _ _ -> v
                     NoParse e    -> error (show e)

parse :: String -> Pos -> ByteString -> RespDerivs
parse tagstr pos s = d
    where d    = RespDerivs flag tag chr pos
          flag = pParenFlags d
          tag  = Parsed tagstr d (nullError d)
          chr  = if BS.null s
                  then NoParse (eofError d)
                  else let (c, s') = (BS.head s, BS.tail s)
                           t = (parse tagstr (nextPos pos c) s')
                       in Parsed c t
                            (nullError d)

[mpscholten]

I think we can close this one now