I think that's not correct. The*installation* doesn't have to be
brand new, the*account* in Thunderbird must be new, i.e. just created.
I added a new New account and could set 'Message Store Type:' to
either 'File per folder (mbox)' or 'File per message (maildir)'.
Hi Yousuf,
When I see things like this, it is usually a failing
drive, especially when the index on teh offending
directory never finishes.
This will show up like a soar thumb if yo run your
drive through gsmartcontrol: check the error logs and
run the self tests
On 4/28/2020 3:14 PM, Paul wrote:
The one I was looking at the other day, said that it didn't
handle stuff in the News folder specifically.
As for the availability of the MailboxStore option in the
Server settings, the claim is that you must use this
immediately when the installation of Thunderbird is
brand new. In my experiments yesterday, I tried to "clean out"
my profile, and tried not to leave any .msf files, then
set the prefs.js with the maildirstore preference, and
that *still* wasn't enough to make it work. I'm going
to have to nuke the damn thing and start from scratch,
to see if I can get it to work.
One other weirdness from yesterdays experiment, is after
I was finished with my failed experiment, I took the ZIP
file holding my unbroken profile, and started to restore
it to my SSD drive. I was greeted by write rates of arounf
2MB/sec on my SSD. It took forever to restore the fleet
of .msf (file per box) style files. And when I opened
Task Manager, MsMpEng was railed on one core, scanning
everything being written into the profile area. I've done
plenty of other stuff on the computer, where it doesn't
do that with quite the same level of venom. (If I unpack
an .ova on a scratch drive, it does that at several hundred
megabytes per second. As if MsMpEng didn't care.)
Paul
Oh, it's a good thing I kept reading the replies, as it looks like you already tried what I was about to try. So it kept using the same file
format as before, even after nuking it and starting from scratch?
Yousuf Khan wrote:
Oh, it's a good thing I kept reading the replies, as it looks like you
already tried what I was about to try. So it kept using the same file
format as before, even after nuking it and starting from scratch?
I would refrain from working in this direction.
Sure, if you have backed up the various folders for TBird
before trying it (like I did when testing), then great.
Just don't do it, without having something to restore from.
It's pretty weird for a function to be existing in TBird
and presumably to be absorbing test time from release to
release, and then be hobbling the usage of it with
inept controls.
On 5/1/2020 3:56 AM, Paul wrote:
Yousuf Khan wrote:
Oh, it's a good thing I kept reading the replies, as it looks like
you already tried what I was about to try. So it kept using the same
file format as before, even after nuking it and starting from scratch?
I would refrain from working in this direction.
Sure, if you have backed up the various folders for TBird
before trying it (like I did when testing), then great.
Just don't do it, without having something to restore from.
It's pretty weird for a function to be existing in TBird
and presumably to be absorbing test time from release to
release, and then be hobbling the usage of it with
inept controls.
I just took a chance, and deleted all of the old newsgroup folders, that contained all of the old-style messages. Left all of the rest of the
files in that news server's base folder untouched. Then I started Thunderbird up again. It re-downloaded the messages, and it only
downloaded from where I last left off. It's now filling the data files
known as *.msf (e.g. alt.comp.os.windows-10.msf) rather than filling the folders! Interestingly, these *.msf files used to exist in this News
folder before, but they were just trivial 1K or 2K files, with nothing substantial inside them. They are now substantial files now, ranging
from 44 KB to 41 MB. So it looks like having those old folders there all
of this time was preventing Thunderbird from using the new style *.msf files, even though it had long ago created them!
Yousuf Khan
I just took a chance, and deleted all of the old newsgroup folders, that contained all of the old-style messages. Left all of the rest of the
files in that news server's base folder untouched. Then I started Thunderbird up again. It re-downloaded the messages, and it only
downloaded from where I last left off. It's now filling the data files
known as *.msf (e.g. alt.comp.os.windows-10.msf) rather than filling the folders! Interestingly, these *.msf files used to exist in this News
folder before, but they were just trivial 1K or 2K files, with nothing substantial inside them. They are now substantial files now, ranging
from 44 KB to 41 MB. So it looks like having those old folders there all
of this time was preventing Thunderbird from using the new style *.msf files, even though it had long ago created them!
<boggle!>
If you apparently did not mind to delete all the old articles, then
why did you keep 580,000 old articles in the first place!?
If you kept your original setup with the 500000 files,
you might try updating to 60.9.1 or so, and trying
to flip the control using that version. It seemed to
unsubscribe me from the one group I'd selected, but
it seems to have worked. I haven't had time to do much
other testing yet.
On 5/1/2020 10:55 AM, Frank Slootweg wrote:
<boggle!>
If you apparently did not mind to delete all the old articles, then
why did you keep 580,000 old articles in the first place!?
Simple, because I had no idea what the purpose of any of these files in
this folder were for in any detail, what was important, and where
exactly data resided, so I just backed up everything. That way I
wouldn't have to recreate everything from scratch, and go through hours
of debugging. I've had situations were just 1 important file goes
missing which screws up the entire configuration, and trying to find
that one missing file among half million is a needle in a haystack.
So now after the deletion, I'm down from half million to only about 600 files. And I did a test backup, and the backup went from over 8 hours,
down to only 2.5 minutes! My feeling is that perhaps a lot of those half-million files were just left over from decades of junk that
Thunderbird did not clear, even though it said it was clearing them.
I can - sort of - understand that, but because these 580,000 were
giving you so much hardship, I would have expected you to look at a
few of them, see that they were just News articles and take it from
there, i.e. set/lower the News retention settings in Thunderbird.
On 4/27/2020 6:15 PM, T wrote:means nothing. test it
Hi Yousuf,
When I see things like this, it is usually a failing
drive, especially when the index on teh offending
directory never finishes.
This will show up like a soar thumb if yo run your
drive through gsmartcontrol: check the error logs and
run the self tests
Brand new drive, less than a month old, hasn't had a chance to get old yet.
Yousuf Khan
On 5/1/2020 1:52 PM, Frank Slootweg wrote:
I can - sort of - understand that, but because these 580,000 were
giving you so much hardship, I would have expected you to look at a
few of them, see that they were just News articles and take it from
there, i.e. set/lower the News retention settings in Thunderbird.
No, I knew those were the message files, considering that there were so
many of them, what else could they have been? But often there are other files interspersed among them, that can often go overlooked because it's overwhelmed by the mass of all of the main files. Just let the backup software handle backing all of it up.
Yes, but "Just let the backup software handle backing all of it up."
was the*problem* which made you start this thread! You can't have it
both ways, it either was a problem/annoyance/<whatever>, or it wasn't!
As to "But often there are other files interspersed among them, that
can often go overlooked because it's overwhelmed by the mass of all of
the main files.", I didn't say to bluntly clobber all the*files*/
*folders*, but to set/lower the News retention settings in Thunderbird.
I.e. let*Thunderbird* do it*safely*, instead of you doing it
(possibly) unsafely.
You*do* know how to set/lower the News retention settings in
Thunderbird, don't you!? (You snipped my other comments about that, so I don't know if you've set/lowered them now.)
I have a folder on one of my SSD drives that takes 8 to 10 hours to back
up. It is only about 1.4 GB, but it is allocated 2.4 GB of space
altogether, and there are 580,000 files here. Indicates that per file
it's using up a little bit over half of a cluster on average. File
system is NTFS.
Meanwhile, this same drive can backup the remainder of the drive in
under 2 hours, and the remainder of the drive is 390 GB! Is NTFS this inefficient for small files like this?
Yousuf Khan
On 4/26/2020 9:24 PM, Yousuf Khan wrote:
I have a folder on one of my SSD drives that takes 8 to 10 hours to
back up. It is only about 1.4 GB, but it is allocated 2.4 GB of space
altogether, and there are 580,000 files here. Indicates that per file
it's using up a little bit over half of a cluster on average. File
system is NTFS.
Meanwhile, this same drive can backup the remainder of the drive in
under 2 hours, and the remainder of the drive is 390 GB! Is NTFS this
inefficient for small files like this?
Yousuf Khan
Okay, so after fixing the problem with my News folder, I kept
researching what these millions of little files were, that were clogging
up my News folder. The files had an extension of WDSEML. Later I found
out that these same files are also there in Email folders, hundreds of thousands of them too.
Initially, I thought that these must be the bodies of the messages that Thunderbird uses to store emails and newsgroup messages. But after a bit
of research, I found out that Thunderbird itself has no use for these
files. Thunderbird does generate them, but it doesn't use them itself. Instead it is generated only for the benefit of Windows' Search and
Indexing application. Windows Search uses it to be able to let you
search messages through the Windows Search box. So once Thunderbird generates these files for Windows Search, it no longer has any use for
them anymore, as it stores its own internal data in a different set of files. In fact, these WDSEML files are saved copies of individual
messages out of Thunderbird's own database. So Thunderbird maintains it
own database, but it never cleans up these copies ever in its life.
WDSEML means "Windows Desktop Search Email", in fact. I also think this
is only a specific problem with Thunderbird under Windows, it probably
isn't an issue in Thunderbird under other OS'es like Linux.
You can easily delete all of these messages, but of course Thunderbird
will regenerate them again as they come in. So what you have to do is
tell Thunderbird not to generate these files for Windows anymore. You go into Thunderbird's options menu and turn it off (Tools → Options, then select Advanced → General → System Integration → Allow Windows search to
search messages).
https://fileinfo.com/extension/wdseml
You can also delete them more easily by searching for and deleting just
the folders in which they reside, rather than the individual files.
These folders have an extension called *.MOZMSGS.
Yousuf Khan
Okay, so after fixing the problem with my News folder, I kept
researching what these millions of little files were, that were
clogging up my News folder. The files had an extension of WDSEML.
Later I found out that these same files are also there in Email
folders, hundreds of thousands of them too.
Initially, I thought that these must be the bodies of the messages
that Thunderbird uses to store emails and newsgroup messages. But
after a bit of research, I found out that Thunderbird itself has no
use for these files. Thunderbird does generate them, but it doesn't
use them itself. Instead it is generated only for the benefit of
Windows' Search and Indexing application. Windows Search uses it to
be able to let you search messages through the Windows Search box. So
once Thunderbird generates these files for Windows Search, it no
longer has any use for them anymore, as it stores its own internal
data in a different set of files. In fact, these WDSEML files are
saved copies of individual messages out of Thunderbird's own
database. So Thunderbird maintains it own database, but it never
cleans up these copies ever in its life. WDSEML means "Windows
Desktop Search Email", in fact. I also think this is only a specific
problem with Thunderbird under Windows, it probably isn't an issue in Thunderbird under other OS'es like Linux.
You can easily delete all of these messages, but of course
Thunderbird will regenerate them again as they come in. So what you
have to do is tell Thunderbird not to generate these files for
Windows anymore. You go into Thunderbird's options menu and turn it
off (Tools → Options, then select Advanced → General → System Integration → Allow Windows search to search messages).
https://fileinfo.com/extension/wdseml
You can also delete them more easily by searching for and deleting
just the folders in which they reside, rather than the individual
files. These folders have an extension called *.MOZMSGS.
Interesting find. I don't remember looking at this option when I
previously trialed Thunderbird. Is this option enabled by default? If
so, a very bad choice my Mozilla.
If I had not known about this option (and I was still using
Thunderbird), and after finding the superfluous and unwanted wdseml
files (since I do*not* want Windows search looking into my e-mails to confuse those hits with those of files where I want to find by name or content), I probably would've added them to the Include option in
CCleaner which I sometimes run manually but is also a daily scheduled
event in Task Scheduler to run before the daily backup. I have other programs that leave shit behind that I want purged, so I go into
CCleaner's options, Include section, and define a template of what to
include in CCleaner's cleanup. Some programs, for example, will save
files for a 'resume' function, like a downloader, to continue the
operation when I next load the program. Nope, if I killed/exited the
program then I do not want it wasting time when I next load the program.
I don't even let my web browsers resume a prior session, and configure
them to purge all local data upon their exit.
To me, having Windows Search dig around inside everything is for those
boobs that are slobs. They haven't a clue how to organize their data,
or are too lazy to do it. They pile thousands of e-mails into the Inbox folder instead of organize the old e-mails into separate pending or
archive folders, and God forbid they delete old e-mails. They'll pile thousands of image files into a single folder instead of use folders to organize them. Foldering is an organizational feature that some users
just seem incapable or unwilling to use. As disorganized is their data
is probably the same for how disorderly is their home.
I'm a bit surprized Mozilla, in developing a cross-platform product,
whould give a gnat's fart about kowtowing to Microsoft's search feature
in Windows. Hell, Mozilla doesn't even use the global certificate store
in Windows (use certmgr.msc to see) within Firefox, and instead uses a private cert store inside of Firefox (and why some programs have to do a double cert install: once into the Windows global cert store and again
into Firefox's private cert store). If users are going to search their
old e-mails, why would they not do that from inside of Thunderbird?
They're searching on e-mails, not on some pic they stored from their
camera or a copy of their tax form. Overreaching got even worse in
Windows 10 with Cortana (which I disabled). All this forensic-like
searching to cater to data slobs.
In the business that would be called a "lazy implementation".
All they would have to do, is write a "search provider" and Windows
could use that to pump the files in an OLE fashion. It could have
been done by making no temporary files at all (flow from MORK file
or MBOX or whatever, right into the Windows.edb, in terms of writes).
I guess there's some benefit to federated search that includes
your email, but to my way of thinking this would only clutter up
a search result later.
You might also discover the Windows.edb file is bloated
beyond recognition, because of that file set. It might
range around 1GB for a vanilla install, but after that
Thunderbird thing got indexed, it would likely double
at the very least.
You can rebuild the Windows.edb index file, using
the Indexing Options control panel in Windows 10.
I would give that a whirl after the TB folder has
had all the cruft removed. It'll take about three
hours to index the regular C: files (but this assumes
you've customized the searched folders to include
most of C: , versus the very shallow folder set used
by default).
Even finding Windows.edb is hard :-) The File Explorer
search won't allow you to find it. You'll need Agent
Ransack or Everything.exe to find that file, just so
you can see the current size, and decide whether it
needs a rebuild or not.
Yousuf Khan <bbbl67@spammenot.yahoo.com> wrote:[...]
You can easily delete all of these messages, but of course
Thunderbird will regenerate them again as they come in. So what you
have to do is tell Thunderbird not to generate these files for
Windows anymore. You go into Thunderbird's options menu and turn it
off (Tools ? Options, then select Advanced ? General ? System
Integration ? Allow Windows search to search messages).
https://fileinfo.com/extension/wdseml
You can also delete them more easily by searching for and deleting
just the folders in which they reside, rather than the individual
files. These folders have an extension called *.MOZMSGS.
Interesting find. I don't remember looking at this option when I
previously trialed Thunderbird. Is this option enabled by default? If
so, a very bad choice my Mozilla.
VanguardLH <V@nguard.lh> wrote:
Yousuf Khan <bbbl67@spammenot.yahoo.com> wrote:[...]
[About .wdseml (Windows Desktop Search Email) files/messages:]
You can easily delete all of these messages, but of course
Thunderbird will regenerate them again as they come in. So what you
have to do is tell Thunderbird not to generate these files for
Windows anymore. You go into Thunderbird's options menu and turn it
off (Tools ? Options, then select Advanced ? General ? System
Integration ? Allow Windows search to search messages).
https://fileinfo.com/extension/wdseml
You can also delete them more easily by searching for and deleting
just the folders in which they reside, rather than the individual
files. These folders have an extension called *.MOZMSGS.
Interesting find. I don't remember looking at this option when I
previously trialed Thunderbird. Is this option enabled by default? If
so, a very bad choice my Mozilla.
No, the option is off by default.
No, the option is off by default.
https://bugzilla.mozilla.org/show_bug.cgi?id=552769
I see someone requested all those .wdseml files (under the .mozmsgs
folders) get deleted if the "Allow Windows Search" option gets disabled. Opened on 10 YEARS AGO! Status is still New. Geezus.
Then at:
https://bugzilla.mozilla.org/show_bug.cgi?id=553048
users try to disable the option but it immediately reenables itself.
Rude! Provide the option but do not honor the user's choice. Yousuf
needs to check if the option: (1) remains disabled across multiple
restarts of Thunderbird; and, (2) if the option remains enabled if the .wdseml files that he deleted are not replace with newly generated
.wdseml files.
https://www.zdnet.com/article/mozilla-we-will-keep-thunderbird-after-all-so-long-as-its-not-a-burden-to-firefox/
"pull the plug with six months' notice if the Thunderbird project does
not make "meaningful progress in short order" in creating technical
infrastructure that's independent of Mozilla Corporation's."
That article is dated back in 2017. So, what magical evolution in development resources has occurred for Thunderbird in the meantime?
"Mozilla stopped throwing resources at the project in 2012"
Somewhat explains why a vast number of big tickets have never been
addressed, but there are tickets still listed as New dating back to
2004. From https://en.wikipedia.org/wiki/Mozilla_Thunderbird#History,
lots of wavering on what to do with this lead balloon.
The more I have dig into Thunderbird and its bugs whether reported or
not, the more I get the feeling that the "developers" are CSCI undergraduates, and over the years the turn over of volunteers resulted
in no old farts left that are intimate with the entire product.
On 12/05/2020 6:25 AM, Frank Slootweg wrote:
No, the option is off by default.
It's off in one section of the options, but it's on in another section.
The more I have dig into Thunderbird and its bugs whether reported or
not, the more I get the feeling that the "developers" are CSCI undergraduates, and over the years the turn over of volunteers resulted
in no old farts left that are intimate with the entire product.
When
Mozilla declared it was considering dumping Thunderbird onto other open-source organizations (like how OpenOffice got dumped at the Apache Software Foundation) was when I decided to terminate my trial of
Thunderbird. Well, that and my exasperation with Thunderbird that
pushed me to also dump it after a 6-month trial.
VanguardLH wrote:
The more I have dig into Thunderbird and its bugs whether reported or
not, the more I get the feeling that the "developers" are CSCI
undergraduates, and over the years the turn over of volunteers resulted
in no old farts left that are intimate with the entire product.
I expect the team is smaller than the one in the photo here.
https://blog.thunderbird.net/2014/11/thunderbird-reorganizes-at-2014-toronto-summit/
Nope, it's off there [1] as well.
I've been using Thunderbird for 10 years and there are no .wdseml
files, no .mozmsgs folders and the 'Allow Windows Search to search
messages' option is unticked in both [2] places.
[1] Where 'there' is as you mentioned:
Tools -> Options -> 'Advanced' major tab -> 'General' sub-tab -> System Integration -> Always check to see if Thunderbird is the default mail
client on startup -> Check Now.... -> In the 'System Integration' popup,
the 'Allow Windows Search to search messages' option is*not* ticked.
[2] It's of course [3] one option, which you can set or unset in these
two places.
[3] OTOH, considering this 'Mozilla' crap, 'of course' is a rather
tricky notion!:-(
Why did it take you *that* long [to abandon the trial of Tbird]! :-)
Out of interest: What are you using now (for e-mail)?
| Sysop: | DaiTengu |
|---|---|
| Location: | Appleton, WI |
| Users: | 1,051 |
| Nodes: | 10 (0 / 10) |
| Uptime: | 246:13:25 |
| Calls: | 13,621 |
| Calls today: | 1 |
| Files: | 186,891 |
| D/L today: |
259 files (86,012K bytes) |
| Messages: | 3,431,553 |