How long did you give it. A whole day is common to rebuild large local stores. I would guess a remote one might take a week. Depending on the upload and download speeds available as the entire contents of each folder will need to be downloaded to index it. It might not be stored, but it is having to arrive so the sorftware can work with it.

Hi!

Thank you for your reply!

It does not look like a function of time. I tried it on a small mail collection, and everything is running on a fast LAN on SSDs. It does not seem like thunderbird is doing anything at all, - there's no visible activity (neither over network nor on the IMAP server which is dovecot in this case). Also, in Tools => Activity monitor there's nothing. The machine is up all the time, and thunderbird is running all the time too.

It hasn't been a week yet though… Though in my understanding, for a mail store with a few hundred of not-so-long messages (with a few Mb total size), it shouldn't take *that* long to complete?

Thanks!

/mjt

Two weeks has passed, thunderbird were running all this time. It definitely had enough time to retrieve multiple copies of the whole thing. It still doesn't find any results within a few folders which are marked to be included in global search results.

How globa *supposed* to work, anyway? Should it include words from message bodies in IMAP folders which are *not* marked as being downloaded (ie, not made this folder available for offline use)?

Thanks,

/mjt

It looks like thinderbird includes in the global search results only those IMAP folders which are marked as made available for offline use *and* as included in global search. Once I make a folder available for offline use, global search starts finding messages in there, but not before. Maybe I'm wrong though.

If this is the case, it will be much more difficult for us, since users has huge amounts of email and keeping two copies of it on the same server is a good free space eater..

My understand it the folder has to be subscribed, but local storage should not be necessary, but that is at odds with your observations and bug comments from decades ago. So I guess I am wrong.

Message search (ctrl+Shift+F) offers to search in the server for non local IMAP accounts and has been around since V2. But details on the gloda implementation are a little scarce. There is this https://bugzilla.mozilla.org/show_bug.cgi?id=517425#c10 or trawling through the source.

Yes, I know all 3 different kinds of search. Globa is a rather unique feature of thunderbird, — this is perhaps one of the top features why we're still with thunderbird despite a few serious bugs which bothers us for 15+ years. Like mailbox (index?) corruption resulting in message filters constantly stopping working, or like duplicating messages in (old) folders - resulting in having 50k dups and counting in a folder with only 5k actual msgs — these are very old bugs reported in bugzilla. This is why we tried to switch to IMAP - thunderbird with local mail storage is barely usable. But the Globa thing is like a drug with good addiction, — so people are still using it despite it makes their health significantly worse ;) I'll think about providing additional storage just to make this work in an IMAP context.

Thank you!

/mjt

Generally I would attribute duple and index corruption to an antivirus. But in a corporate environment file contention comes from many other sources. Thunderbird was originally written to run in a single process environment it does not handle file contention at all really. So to maximize reliability things have to be configured around it to try and reduce contention to Zero. So, no streaming backups, no cloud synchronization. Take extreme care when storing profile data anywhere but on a local disk, but above all prevent antivirus programs from scanning in the Thunderbird profile folder "on demand". I recommend an exception in antivirus product for the Thunderbird profile folder. Likewise an third party toolbars in Thunderbird usually result in at least index corruption. Especially those scam and spam tools.

Yes they are old bugs, but most of them are not going to be fixed until the maildir like store becomes the default store. Multi gigabyte storage files per folder are the real issue for most of the bugs, exacerbated by external factors.

Personally I use Thunderbird on a single PC with a 40Gb profile folder collection and it works pretty well. I can not even begin to say it is barely usable as you claim.

I am positively tired of it hanging all the time in V115. I hope V128 has fixed that. They say they have fixed most of the issues. We will see.

But the topics you mention are mostly manageable, but it does require doubting the lies antivirus vendors promote about what are real risks v's what are potential ones, but I got used to doubting everyone 20+ years ago when Microsoft started their total cost of ownership stuff which was largely fiction and fear.

There's no contention here, and no antivirus either. We had thunderbird mail store on a file server (samba) - this is to avoid it being in windows user profile where it does not belong at all. But I watch very carefully for the users to never run two instances of thunderbird with the same mail storage — for this I only allow users to log in to a single PC. There's *zero* contention, and we had thunderbird mail storage folders added to AV exclusions many years ago too.

Both problems I mentioned in the previous message only occurs when email is on the network drive though, — and the bugs in bugzilla also mentions network drive (tried to find them once again - I remember finding them a few times in the past - but can't currently). We have this setup for backup purposes (backups are done on server only) and to be able to switch to another PC, so mail location is not dependent on the PC the user is working on, just like windows roaming profiles (eg, to replace a PC and have the same setup for users).

When the whole thing is on local drive everything seems to be fine (except maybe another issue which sometimes bothers us — with thunderbird doubling the leading ">" and hence sometimes corrupting program code embedded in text messages — I also hoped this will be fixed by switching to IMAP, but since it has to download all mesages into mbox files anyway for globa to work, this issue wont be fixed anyway).

I don't know if this is a prob in samba or windows file client, or something else.

Thanks,

/mjt

Samba is known to have issues. Sorry, I cannot list them but I would not fully trust it.

The indexing for global message search might get stalled on ill-formed messages, after which point a folder will stop being indexed.

Regarding performance (I can't tell what you have and have not tried):

• samba will make things worse
• antivirus exclusions are a must
• version 128 coming up has significant improvement when messages are downloading

Yeah, I'm switching from samba. It'd be *very* nice to have samba fixed in this context. It's used too widely.

Indexing does not stall on bad messages here, it works just fine when all messages are downloaded locally (hence my original question - if globa works on IMAP folders which are not made available locally).

I didn't have questions about performance. Yes I know thunderbird performance is much worse on samba than on local drive, even if network SSD speed is significantly better than local HDD — it must be a way how thunderbird does the I/O.

Antivirus actually had never been a problem for us. I turned it off for some users mail storage, but not for all users (message duplication and other funny things happens independently of antivirus; here I'm testing with antivirus exception).

I look forward for v128; though it does not seem like it will address any of the real issues which bothers us (indexing remote messages in globa without storing them locally is the main one).

Thanks,

/mjt

So, is thunderbird supposed to index IMAP messages without keeping whole message text locally at the same time?

On a newly created profile for an existing IMAP account, with "Store all messages locally" initially turned OFF, thunderbird will build globa for quite some time, saying it is indexing this or that folder in the "Work Indicator" window. Judging on the speed of this process, it looks like thunderbird is downloading whole messages - at least it takes significantly more time to "Index" each folder than to load message headers.

Yet, when searching for any word, thunderbird only finds this word in subjects, not bodies of the messages.

So it looks like there's something wrong here. On one hand it looks like TB is trying to build a real global index. On the other hand it does not find message bodies using this index, only message subjects.

Thank you!

Let's back up. Gloda (with a "d") is what provides global search - ctrl+K. https://support.mozilla.org/en-US/kb/global-search

For global search to provide full search results on message body, yes, it is necessary for the entire message to be downloaded.

If you are using imap, you must have enabled Message Synchronizing in Account Settings, for all folder, for all accounts.

If you are using pop, you must not have enabled Fetch Headers only in server account settings.

Okay, that is what I observe as well. Thank you for the reply!

This brings 2 more questions.

1. Why TB is "indexing" all the messages when Gloda is enabled but Message Syncronizing is not? It definitely fetches all message *bodies* in this case, - I looked at IMAP conversation. First it downloads all headers to build folder index, and next it downloads whole messages in each folder, "Activity Manager" window shows "Indexing messages in $Folder" at this time. It seems such downloading is useless since the results aren't used anyway.

2. Shouldn't the "Include messages in this folder in Global Search results" checkbox in the folder properties be disabled (and unchecked) if "Select this folder for offline use" (which is a folder-level form of "Message Syncronizing" checkbox) is unchecked? This checkbox is definitely confusing, - if this folder is not selected for offline use, but "Include this folder in Gloda" is checked (by default!), it's surely expected that the folder to be searched in.

Both of these 2 questions are the reasons why I started this thread: it looks like TB should be indexing all messages in Gloda even if "Message Syncronizing" is turned off, and it seems to be indexing them and each folder is included in Gloda in folder properties. But the search isn't working anyway.

What it is indexing, why it downloads all message bodies anyway?

Thank you!

/mjt