Limited write cycles on Flash memory cards

[Marius]

Hi,

 

In the past I have heard several times there is a serious limitation of write cycles on Flash memory cells.

 

Ofcourse 10.000 or 100.000 times or whatever between it... sounds as very much, but there is a small problem:

 

when you fill a 16MB partition, loaded with atari files, the files are stored one-by-one, so for every file the VTOC sectors are updated.

 

When I fill my 16MB partition the boot sectors of this ATR are updated thousands times!

 

I was wondering if this kind of writing is also limited to that virtual 10.000?

 

Is there a difference between writing/updating and erasing?

 

I've also heard that formatting is pulling the cell back to ZERO, and erasing is less 'aggressive', so erasing/writing/updating could be done much more than real formatting.

 

who knows more?

Marius

[Rybags]

A "format" just re-initializes the file system in most cases. So, it isn't actually erasing very much at all.

 

Sounds right about copying to a virtual drive though. Really, the best method (if possible) would be to create such a virtual ATR image on a PC, then just dump it down to the memory card in one operation.

 

Flash memory is similar to the older type EPROM in that it can only be "erased" to bitstates of 1 - writing just involves telling it what bits need to be 0.

 

So, a write operation will occur at a block level, and involves an erase operation as well.

 

Some flash devices can perform block reallocation at the firmware level in the event that blocks fail - I don't know if that applies to SD/XD type cards though.

 

There's a reasonably good article in Wiki about Flash memory technology.

[Marius]

Sounds right about copying to a virtual drive though. Really, the best method (if possible) would be to create such a virtual ATR image on a PC, then just dump it down to the memory card in one operation.

 

thanks for you info...

 

Your solution is cool for Sio2IDE and Sio2SD... but at this moment I'm writing back my ATR to MyIDE :s....

 

I know there is a tool to write MyIDE CF cards too by PC, but I don't think that tool works with 16MB partitions.

 

but your solution is cool indeed.... I'll think that over.

 

Marius

[HiassofT]

Hi Marius!

I know there is a tool to write MyIDE CF cards too by PC, but I don't think that tool works with 16MB partitions.

My myidetool should be able to read/write 16MB partitions. At least, last time I tested it it worked fine. If it doesn't work anymore, a bug might have sneaked in.

 

Concerning flash write cycles: Almost all vendors use some kind of sector-remapping but they don't tell you how many spare blocks they use or how the algorithm works. IIRC the german computer magazine c't did a test where they wrote to the same block again and again. Even after a really large number of writes the whole flash drive still worked perfectly well. Without further implementation details it's really hard to estimate how long a flash drive will work, but for most cases we can assume it'll be much longer than the 10k/100k writes (unless the vendor didn't implement any remapping at all).

 

so long,

 

Hias

[+Larry]

Hi,

 

In the past I have heard several times there is a serious limitation of write cycles on Flash memory cells.

 

(snip...)

 

who knows more?

Marius

 

I'm not sure if this will address your concerns or not, but...

there is a wealth of info on the internet about this subject. As mentioned, virtually all mfgs. of flash devices use "wear leveling" to help preserve data integrity. And like most things in life, not all flash memory is created equal. Stick with highly-rated device makers such as SanDisk and Toshiba. If you can, purchase "industrial-rated" devices, although this is getting harder to do as the distinction between "industrial" and "consumer" blurs. Some industrial-rated devices are rated to up to 2,000,000 write cycles. Their algorithms are proprietary, but you can guess that higher-rated devices have more agressive re-mapping and many more spare cells/blocks. Since flash devices are getting incredibly cheap, you can also do your own "wear leveling" by moving your MyIDE partitions within the device. Of course, it is just good practice to back up your flash (or any other) hard drive -- I sector copy mine at least once a month to an APE drive on my PC and/or a completely separate USB hard drive used only for backups. But I've been using flash drives for several years and never lost a thing due to bad flash memory.

-Larry

[JamesD]

I've already had USB FLASH drives die but I can't be certain it was the FLASH chip that's the problem.

Many of these devices are overrated as to writes. Kinda like the MTBF ratings of hard drives. In the real world they tend to experience a much higher failure rate than the manufacturers advertise.

However, there are also chips that will have a longer life span so if you buy the right drive you shouldn't have a problem.

With the price of the devices there's no reason you couldn't buy a couple and back up everything periodically.

[Marius]

Thanks guys for all your kind replies.

 

Best conclusion for me personal is I should not copy file-by-file to a 16MB partition. I should setup the partition first on a NON-flash partition (or on PC, or on atari harddisk) and when it is finished, it should be stored as one file on a Flash device.

 

And when there are any import things, backups are your friend (but that was already known haha)

 

file-by-file copy is slow and not needed. Pfff yesterday I backed up 1560 files from 16MB atr to 16MB myIDE partition ... using standard SIO speed, file by file (with a tool called 'hardback') ... that is asking for trouble ofcourse. It took the atari over 3 hours to copy all that files.

 

@hiassoft: I have to test your tool. Could you give me a link to your site where I can find the most recent version?

 

EDIT: Forget about the link hias, I have your tool. And it is outstanding!!! What a fantastic thing. It is amazing, this increases functionality of MyIDE! Thanks. (Btw. extracting a 16MB partition takes a lot time, is this due of my USB 1.1 Card Reader, or is it also slow on USB 2.0?)

 

Thanks a lot.

This forum is really a great resource of help and info to me!

Great.

 

Marius

Edited September 10, 2007 by Marius1976

[HiassofT]

EDIT: Forget about the link hias, I have your tool. And it is outstanding!!! What a fantastic thing. It is amazing, this increases functionality of MyIDE! Thanks. (Btw. extracting a 16MB partition takes a lot time, is this due of my USB 1.1 Card Reader, or is it also slow on USB 2.0?)

With USB 2.0 it's quite fast, but USB 1.1 is really slow. I bought a USB 2.0 controller card for my old computer and everything was a lot better then.

 

BTW: It could also be the USB/CF adapter. I got an el-cheapo one and never got the ~8-10 MB/sec as advertised in the CF card specs.

 

BTW2: also remeber that myidetool has to transfer twice the amount of data, since myide usually "throws away" half of the space.

 

BTW3: I'm currently working on adding support for Shawn Jefferson's Image Manager. Using images with MyIDE + image manager + myidetool should then get a lot easier, too :-)

 

so long,

 

Hias

[a8isa1]

EDIT: Forget about the link hias, I have your tool. And it is outstanding!!! What a fantastic thing. It is amazing, this increases functionality of MyIDE! Thanks. (Btw. extracting a 16MB partition takes a lot time, is this due of my USB 1.1 Card Reader, or is it also slow on USB 2.0?)

With USB 2.0 it's quite fast, but USB 1.1 is really slow. I bought a USB 2.0 controller card for my old computer and everything was a lot better then.

 

BTW: It could also be the USB/CF adapter. I got an el-cheapo one and never got the ~8-10 MB/sec as advertised in the CF card specs.

 

BTW2: also remeber that myidetool has to transfer twice the amount of data, since myide usually "throws away" half of the space.

 

BTW3: I'm currently working on adding support for Shawn Jefferson's Image Manager. Using images with MyIDE + image manager + myidetool should then get a lot easier, too :-)

 

so long,

 

Hias

 

Sorry, off topic but can linux dd be used to archive a MyIDE drive or flash module?

 

Thanks.

 

- Steve

[supercat]

It's unfortunate, IMHO, that nearly all flash-file-system products work by having the flash mimic a sector-based storage medium, and then stick a file system on top of that (using the sector-based paradigm for all operations). This approach hampers performance in many cases, since the device handling the sector remapping has no way of knowing which data are meaningful and which are not.

 

For example, suppose it's necessary to overwrite on sector on a block that happened to contain 32 seemingly-valid sectors worth of data. Before that block can be reused, the flash drive will have to copy all 32 sectors to other blocks. Even if most of those sectors represent files that have been deleted (and it would thus be just fine to ignore the data therein), the sector mapper won't know that. Flash cards often get slower and slower with age not because of any inherent electrical failing, but because they become cluttered with lots of old meaningless data that they have to keep shuffling around.

[HiassofT]

Hi Steve!

Sorry, off topic but can linux dd be used to archive a MyIDE drive or flash module?

Yes, you can use this to do a full backup.

I also use this method to test myidetool: I created an image of my 512MB CF card and then use myidetool with the image instead of the CF card.

 

so long,

 

Hias

[pseudografx]

Flash cards often get slower and slower with age not because of any inherent electrical failing, but because they become cluttered with lots of old meaningless data that they have to keep shuffling around.

Is there a way to reinitialize them, e.g. by some kind of low-level format?