The key trait of consumable items are that players can only reap their benefits for a limited number of times. It might be a one-use item, or an item with a limited number of charges, or even something that can only be activated so often (once a day, once a level, etc). When done right, they become a precious resource for the players to manage, a secret weapon to hold onto until the moment is right, or a last resort that might solve the current problems facing the party, but whose consequences might prove even more perilous.
Ace in the hole
The most common use for consumables in rogue-like games is as a sort of "get out of jail" card. There are potions that let you heal yourself, scrolls that let you immediately teleport out of danger, and wands which blast your foes with power far beyond what a character of your level would usually be able to muster. All of these effects are incredibly powerful, and would be terribly unbalancing in some circumstances if they were permanent or regular abilities, but because they have a limited number of uses they instead become things the player hoards until he gets overwhelmed, and then falls back on to catch his breath and hopefully (but not always) live to fight another day.
The fact that these powers are all limited means that they can't truly "break" the game -- yes they may trivialize a single encounter, but in the grand scheme of things that's usually enough to back out of one bad decision, or to brute-force your way through an otherwise difficult situation. After that, the player is left alive but unmistakably weakened -- his arsenal depleted. This balance of efficacy and rarity in the form of a "power reserve" serves an important role in rogue-likes as a sort of "buffer" for when the player bites off more than he can chew (or the random generator does something particularly malicious). In effect, the player has a sort of "overdrive" that the consumable items allow which lets them take on challenges that should out-match them, or makes escape and survival actually viable in cases where it's clear a head on fight would be hopeless. It makes the "fairness" of the content the game generates less important (in an absolute sense) because the player should have the tools he needs to survive -- provided he manages to ration them wisely.
The same is true for D&D. Giving players an endless stream of healing potions and one-shot items greatly reduces the difficulty of the game and makes consumables just another think to keep track of; however, making them precious resources (and making sure to push players occasionally so they need to use them) means that players must constantly evaluate whether they have it in them to push on, or if they need to burn some of their power reserve to win the day. Interestingly, Dungeon Crawl Classics incorporates a mechanic very similar to this by letting players literally "burn" various stats, sacrificing ability points in the long term for an immediate short-term boost in power. A fighter might make a desperate push at the cost of physical over-exertion, a wizard might channel so much energy into a spell that he risks becoming permanently warped, a rogue may... well, push his luck too much I guess. Still, the net result and impact is very similar, and makes not only challenging but dynamically generated content more viable by giving players the ability to deal with extreme difficulty, but giving meaningful consequence for using such a resource.
The fact that these powers are all limited means that they can't truly "break" the game -- yes they may trivialize a single encounter, but in the grand scheme of things that's usually enough to back out of one bad decision, or to brute-force your way through an otherwise difficult situation. After that, the player is left alive but unmistakably weakened -- his arsenal depleted. This balance of efficacy and rarity in the form of a "power reserve" serves an important role in rogue-likes as a sort of "buffer" for when the player bites off more than he can chew (or the random generator does something particularly malicious). In effect, the player has a sort of "overdrive" that the consumable items allow which lets them take on challenges that should out-match them, or makes escape and survival actually viable in cases where it's clear a head on fight would be hopeless. It makes the "fairness" of the content the game generates less important (in an absolute sense) because the player should have the tools he needs to survive -- provided he manages to ration them wisely.
The same is true for D&D. Giving players an endless stream of healing potions and one-shot items greatly reduces the difficulty of the game and makes consumables just another think to keep track of; however, making them precious resources (and making sure to push players occasionally so they need to use them) means that players must constantly evaluate whether they have it in them to push on, or if they need to burn some of their power reserve to win the day. Interestingly, Dungeon Crawl Classics incorporates a mechanic very similar to this by letting players literally "burn" various stats, sacrificing ability points in the long term for an immediate short-term boost in power. A fighter might make a desperate push at the cost of physical over-exertion, a wizard might channel so much energy into a spell that he risks becoming permanently warped, a rogue may... well, push his luck too much I guess. Still, the net result and impact is very similar, and makes not only challenging but dynamically generated content more viable by giving players the ability to deal with extreme difficulty, but giving meaningful consequence for using such a resource.
Planned obsolescence
The constrained nature of consumables gives the designer/DM a lot of freedom to be creative. With powers that can only be invoked once, the amount of harm that an interesting but potentially unbalanced item is tightly limited. Consumable items can be incredibly powerful, full of flavor or sub-mechanics that you want to experience, or a way of granting players a very limited "deus ex machina" ability without opening the flood gates.
For this reason, consumable/limited items make excellent loot, since they allow the DM to make them very powerful and memorable without risking them ruin his game. If you give the player a scroll of "Power Word: Kill" you should expect them to use it eventually, and expect them to feel really awesome when they do. Don't feel bad if they use it to thwart a choice encounter, just throw enough stuff at them that they have to really earn it by doing well enough to keep it in reserve until then.
Another interesting component of limited items in rogue-likes is that they're not interchangeable. Healing potions are the baseline way to add a little extra survivability, but they will be of little help when being chased down by a monster where you really need teleportation, or if you need to deal with a hydra or crystal statue that you dare not get close to and have no other good means of fighting besides a wand of disintegration. Instead of just giving players various flavors of "this flask of alchemist's X does 5 points Y damage in burst 1" you get a bunch of different tools that have very different functions. Healing potions give you a boost of survivability, teleportation scrolls give you a chance for escape, wands let you use abilities you might not otherwise have access too -- and all are called on for in different situations. For this reason, consumable/limited items make excellent loot, since they allow the DM to make them very powerful and memorable without risking them ruin his game. If you give the player a scroll of "Power Word: Kill" you should expect them to use it eventually, and expect them to feel really awesome when they do. Don't feel bad if they use it to thwart a choice encounter, just throw enough stuff at them that they have to really earn it by doing well enough to keep it in reserve until then.
Variety is the spice of life
By giving players diverse limited items, you give them extra power, but still preserve the sense of limitation. Players will be much more reluctant to use their last healing potion, or save that last charge in the wand of disintegration until they really need it, forcing them to vary up their own usages and strategies. Making resource management about options instead of limits makes it a lot more fun and interesting.
Limits come in all forms
Finally, limited items don't just have to be restricted to potions and charged items. As mentioned before, Dungeon Crawl Classics has an interesting variation involving sacrificing ability points. The rogue-like game Dungeon Crawl: Stone Soup has an innovative take on magic items where certain effects can increase ones "magic contamination". The effects (invisibility, powerful buffs like haste, etc) are incredibly useful; however, if ones contamination rises above a certain level by using such effects for too long before letting the contamination fades, the player runs the risk of acquiring one of several harmful mutations (stat drain, growths that don't let you wear armor, other very nasty traits). If you feel that you're about to die it might be worth the risk; however, it's certainly perilous enough to discourage casual use. Such a system would be great in a low-magic setting where arcane magic exists but is feared, and dangerous. Mages are twisted not because magic is for the evil, but because magic itself twists you, and while you might be tempted to wear that +2 circlet of intelligence, doing so for too long could lead to your irrevocable corruption. This approach turns what would usually be a simple case of smashing the loot pinata into a grueling, soul-searching question of risk and reward.
And isn't that what role playing games are all about?