Lépjen offline állapotba az Player FM alkalmazással!
428: Ruminating on Ruby Enumerators
Manage episode 422958694 series 1401614
Joël explains his note-taking system, which he uses to capture his beliefs and thoughts about software development. Stephanie recalls feedback from her recent RailsConf talk, where her confidence stemmed from deeply believing in her material despite limited rehearsal. This leads to a conversation about the value of mental models in building a comprehensive understanding of a topic, which can foster confidence and adaptability during presentations and discussions.
The episode then shifts focus to the practical application of enumerators in Ruby, exploring various mental models to understand their functionality better. Joël introduces several metaphors, such as enumerators as cursors, lazy collections, and sequence generators, which help demystify their use cases.
- Episode on note-taking
- What we believe about software
- Ruby Enumerators
- Enumerator Lazy
- Modeling a Paginated API as a lazy stream
- Solving a memory performance issue with enumerator
- Find in batches
- Binary tree implementation with different traversals
- Teaching Ruby to Count
Transcript:
STEPHANIE: Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Stephanie Minn.
JOËL: And I'm Joël Quenneville, and together, we're here to share a bit of what we've learned along the way.
STEPHANIE: So, Joël, what's new in your world?
JOËL: So, what's new in my world isn't exactly a new thing. I've talked about it on the podcast here before, and it's my note-taking system. I have a system where I try to capture notes that are things I believe about software or things I think are probably true about software. They're chunked up in really small pieces, such that every note is effectively one small thesis statement and a paragraph of text, and maybe a diagram or a code snippet to support that. And then, it's highly hyperlinked to other notes. So, I sort of build out some thoughts on software that way.
A thing that I've done recently that's been pretty exciting with that is introducing a sort of separate set of notes that connect to my sort of opinion notes. So, I create individual notes for public works that I've done, things like blog posts or conference talks. Because a lot of those are built on top of ideas that have been sitting in my note system for a while. Readers and listeners get to sort of see the final product, but often sort of built up over several months or even a couple of years as I added different notes that kind of circled a topic and then eventually got to a thing.
What I did, though, was actually making those connections explicit. And so I use Obsidian. Obsidian has this cool graph view where it just sort of shows all of the notes, and it circles them with, like, connections between them where the notes connect. So, I can now see in a visual format how my thoughts cluster in different topics, but then also which clusters have talks and blog posts hanging off of them and also which ones don't, which ones are like, oh, I have a lot of thoughts on this topic, and I've not yet written about it in a public forum; maybe that would be a thing to explore. So, seeing that visual got me really excited. I was having a good time.
STEPHANIE: Yes, I have several thoughts coming to mind in response, which is, I know you love a visual. I really like the system of, even if you have created content for it, like, you have a space for, like, thoughts about it to evolve. Because you said, like, sometimes content comes out of notes that you've been...or, like, thoughts you've been having over years, but it's like, even afterwards, I'm sure there will still be new thoughts about it, too. I always have a hard time finding a place for that thing kind of once I, I don't know, it's like some of that stuff is never really considered done, right? So, that is really cool.
And I also was just thinking about an old episode of The Bike Shed back when Chris Toomey and Steph Viccari hosted the podcast called "What We Believe About Software," I think, is the title. And I was just thinking about how, like, if only we could just dump all of your notes [laughs] into some, you know, stream [laughs], and that would be really cool. If we ever do, like, an episode like that, that would be really fun. And I'm sure, you know, you already have this, like, huge bank of ideas [laughs].
JOËL: Yes. It is really fun because I build up...the thoughts are often sort of interconnected, and so they might have a topic, but they are very focused. So, I might have, like, three or four things I believe about a particular topic that cluster together. So, we could...and, actually, I have used, in the past, some of those clusters as initial food for thought for a Bikeshed episode.
STEPHANIE: Yeah, that's really neat. I like this idea of a kind of just, like, a repository for putting down what you believe about software as kind of, like, guiding principles for yourself as a developer a little bit.
I remember a piece of feedback I got about my RailsConf talk that I gave a few weeks ago, and someone said like, "Oh, you sounded really confident in what you were talking about." And that surprised me because I, like, didn't practice rehearsing giving the talk all that much [laughs]. It's because they had asked like, "Oh, like, did you practice a lot?" or something like that. And I think I realized that I, like, really believed in what I was sharing and kind of that, I think, was perhaps what they were picking up on.
And even though, like, maybe the rehearsal of the presentation itself was not where I had spent a lot of time on, I had spent a lot of time thinking about what I wanted to share and just building up my confidence around that. So, I thought that was an interesting connection.
JOËL: Yeah, you fully developed the idea. You kind of explored all the side trails, maybe a little bit on your own as well. You're on very familiar terrain. And so, that is a way of building confidence separate from just sort of memorizing a talk.
STEPHANIE: Yeah, yeah. Exactly.
JOËL: In a sense, I almost feel like that's a better sense of confidence because then you can sort of...you can roll with the punches. You know, if a slide is out of order or something, sure, it maybe messes up a little bit of the narrative that you're trying to say. But you're not like, "Oh no, what is this content?" You're like, "Oh yeah, this thing," and you can dive right into it. Somebody asks you a question, and you're not like, "Oh no, that was not in the script," because, again, you've sort of mastered your topic. You know the area as a whole, even sort of the blurry edges beyond the talk, and can react in a way that is pretty confident.
STEPHANIE: Yeah. I still definitely fear the open Q&A. I've never done it before, but maybe one day I will be able to because I just, you know, know my topic so well inside and out [laughs] that I can roll with the punches, as you say.
JOËL: Open Q&A is just...it's a roll of the dice. Sometimes, you get some really good conversation topics there, and sometimes, it's just a waste of everyone's time.
STEPHANIE: I like that take [laughs].
JOËL: Maybe that should go into the things I believe about software. So, other than receiving feedback about your RailsConf talk, what is new in your world?
STEPHANIE: Yeah, so I am wrapping up a pretty large project on my client work that we're hoping to release soon. And, in fact, it's actually being released along with a big announcement from the client company to their customers. Essentially, at a conference, they're going to say like, "Hey, like, we now have this new feature." And so, I think there's some hype generated around it. And this past week, we've been doing a lot of internal testing of the feature because there are a lot of employees of my client company who are, like, pretty big users of the product, which is cool because I think we're getting, you know, we have easy access to people who can give us good feedback.
But I am having a hard time with being on the receiving end of the feedback and figuring out, like, what is stuff I need to attend to now before, you know, this big release? And what is stuff that is just kind of, like, general feedback like, "Oh, like, I wish it did this," but, you know, it turns out that that's not really what we were building? And how do I just kind of, like, accept that?
You know, it's coming from a good place, but I can't really help them there, at least right now. And that's hard for me because I like helping people, right? And so, if someone says something like, "Oh, like, I wish it did this," or like, "Oh, that's kind of weird," I'm like, "Oh, I want to just, like, fix that for you right now [laughs]." And I suspect that a lot of other devs can relate to this, especially if, like, you know, you've been working on something for a little bit, and it feels...I'm just going to say it: it feels a little precious to me.
So, what I'm trying to do today, actually, is not look at any of the feedback at all [laughs] and come at it tomorrow with a bit of a calmer vibe and be able to separate out, like, you know, I think all feedback is informative, but not all of it is useful for you at any given moment. Like, if there are bugs, then those will be my immediate priority. If there's maybe some small tweaks that we can make the feature just a little bit more polished, then I also think those are good.
But then we are discovering a few things, too, about, like, what this feature is or could be. And I think those are the things that, you know, need to be brought into a conversation with a broader group and think about, like, is this the direction we want to go? So, that's kind of how I'm bucketing that feedback right now.
JOËL: How do you feel about receiving direct feedback versus having something filtered through something like a product team?
STEPHANIE: Ooh, that's an interesting question. Because right now we're doing, I think, a mix of both that I'm not sure that I really like. On one hand, when it's filtered, it's hard to get to the root of what someone is asking for. And oftentimes, like, it may not even include enough information after the fact to be able to come at it from a dev perspective. But then direct feedback, I think, is just a little bit overwhelming sometimes. And it can be hard to figure out what to pay attention to if you don't have that, like, input from a product team about, like, what the roadmap is looking like or where, you know, strategically their heads are at.
So, one thing that kind of has emerged from this is like, oh, I was getting, you know, notifications for the feedback coming in. And what we did was set up a meeting [laughs] so that we can...maybe all of us can, like, scan it together ahead of time and then come at it with a little bit of context about what's come in but then maybe coalesce around the things that we feel are important.
JOËL: Well, you'll have to keep us updated on how that plays out, and we can kind of hear what is the balance that ends up working well for you.
STEPHANIE: Yeah, I hope so. I think this is actually maybe something that's a bit underexplored from the dev perspective, you know, that in-between stage of you're not totally done because it's not shipped to the world yet, but, you know, you're starting to get a little bit of that input. And what you do with that? Because I think there is some value in being engaged in that process.
JOËL: So, we were talking earlier about this note-taking system that I use and sort of a renewed excitement that I have about it. And one thing that I did when I was going through and finding clusters of things that hadn't been written about was I found that I had a cluster of notes on different mental models that I had for understanding Ruby enumerators, not the enumerable module, but the enumerator object. And I decided, you know what? This would probably make for a good blog post. So, I drafted a blog post, and I've been thinking about this a little bit more recently. So, I've been really hyped about digging into enumerators because of that experience.
STEPHANIE: Yeah, that's very cool. I have to say that I feel like I did not know a lot about enumerators and the API for them kind of before you brought this topic up, and I did a bit of a deep dive in preparation for us to discuss it. I feel like most devs, you know, work with enumerators via methods on enumerable without totally knowing that they are. So, I think that this would be a really interesting episode for people to be like, oh, like, I've been using this stuff, you know, the whole time, and now I can have a different perspective or just more insight on what they can do.
JOËL: Before we dig into individual mental models, though, I want to think a little bit about the concept of mental models as a whole. Years ago, someone gave me advice to sort of pay attention to mental models, ways I think about the world or different code structures, different code approaches, and that really stuck with me. So, I've since been, like, kind of, like, collecting mental models.
And, in a way, they're like a, for me, a bit more of a concrete way to look at a particular topic. So, I can say I'm looking at this particular topic through the lens of a particular mental model that helps me build more clarity around it. And if I have three or four, then I can kind of look at it from three or four different perspectives. And now, all of a sudden, I feel like I'm seeing in three dimensions.
STEPHANIE: Whoa, the Matrix even [laughs]. That's cool. Yeah, I really like that advice. I think I'm going to steal it and start kind of suggesting it to other people because I think, in a way, on this show, that has come through a lot. And talking about things on the podcast has helped me develop a lot of my mental models. And I think we've done a few, like, episodes in the past about various ones we have for just our work because it's like, that's infinite [laughs]. But what I really have been appreciating is that mental models just need to work for you. As long as you're able to understand something, then it's valuable.
And that has really helped me also, like, just get on the same understanding with others because the goal is not necessarily to, like, explain it the way that I would think of it, but figure out what would help them kind of develop their own mental model for understanding something, and, you know, kind of as long as we both feel like we have that shared understanding, no matter what lens it's through. And, you know, sometimes it's even more effective when we are able to share it. But I feel like, you know, you can still find ways to collaborate on something with a diversity of mental models.
JOËL: Yeah, they're a great way to build self-understanding. They're a great way to sort of build understanding between two people. So, I'm a huge fan of the concept. And part of what I've been doing with my note-taking system is trying to capture those as much as possible. If I'm ever, like, trying to understand a complex topic and I'm like, oh, I think I've got a breakthrough here; I understand it; it's kind of like this, or you can imagine it in this perspective, it's like, write that down. That's gold.
STEPHANIE: Very cool. So, Joël, would you be able to share some of your mental models for enumerator?
JOËL: So, one way that I look at it is the idea that an enumerator is effectively a cursor over a collection. So, you have an array and a regular array; you're either in the middle of iterating through it using something like each, or you're not. You just have a collection of items. Enumerator introduces the idea that you're actually sort of at a position in the array. So, you're sort of focused on, let's say, the third item or the fourth item. You have a cursor there, and you can move that cursor forward as you sort of step through.
But the really cool thing is you can also kind of pause and just pass that cursor on to someone else, and someone else can move the cursor a few steps further down the collection, pause, pass it on to someone else. And it's totally fine. Nobody has to, like, go through an entire, like, each iteration.
STEPHANIE: Yeah. So, when you were talking about cursors, that got me thinking a little bit because I actually have struggled with that concept, especially when it comes to, you know, things code-related. Like, when I've had to work with database things and stuff, like, the idea of a cursor was a little, like, difficult for me to wrap my head around. And I was looking at the methods on enumerator, like the instance methods on enumerator. And one of them actually is what helped me develop this mental model. And I'm excited to see what you think.
But there is a rewind method that basically rewinds the sequence back to its beginning, right? And what that triggered for me was a VHS tape [laughs] and just those, like, car-shaped rewinders for tapes back in the '90s. I don't know if you ever had one in your house, but I did. And I just thought that was such a cool method name because it was very, I don't know, it was just like a word that we use in the English language, right?
So, the idea of, like, tapes, you know, like, cassette tapes or VHS tape kind of also it sounds like it matches well with what you were sharing, too, where it's like, I could pass, I don't know, maybe I, like, listen to a few songs on my cassette tape, and then I give it to someone else, and they can pick up where I left off. And yeah, that was really helpful in understanding, like, a marker of a position a little more than cursor was able to for me.
JOËL: That's really interesting because now I wonder, like, how far we could push that metaphor. So, musical data is encoded on magnetic tape. Cassette tapes typically there are sort of two spools. You start off with all of the tape wound up around one spool, and then as it sort of moves across the read head, it gets wound up on sort of the, I don't know, destination spool. I guess you can call them origin and destination. And because of that, you can sort of be in a, like, partly read state where, you know, half the tape is on the destination spool, half of it is on the origin spool, and you have that read head that's in the middle, and you're just kind of paused there. And you can kind of jump forward in that.
So, I imagine something like that in your metaphor is like an enumerator. Contrast that to imagine just a single spool, which is just we have musical data encoded on magnetic tape, and we wrapped it up on a spool. I feel like that's almost more like a regular array because you don't have that concept of, like, position, or being able to read parts of it or anything like that. It's just, here's some data.
STEPHANIE: Yeah. While you were talking about the two spools, I was thinking about, like, part of what is nice about enumerator is that you can go forward or backwards, right? And that feels a little more possible with that two-spool metaphor [laughs], rather than just unraveling something, where you are kind of discarding what has already been read.
JOËL: The one caveat there is that enumerators can move forward one item at a time. They can only move backwards by jumping back to the beginning. So, you can't step forward or step back.
STEPHANIE: Yeah, that's fair.
JOËL: You step forward, or you, like, rewind to the beginning. I think, in my mind, I was thinking a little bit more about this metaphor. And I think it's also just a metaphor for what's called the External Iterator Pattern. It's one of the classic Gang of Four Patterns, which is what enumerator, the object in Ruby, is an implementation of. I feel like I always see that in the documentation, like, oh, enumerator is an implementation of the External Iterator Pattern. And I just kind of go, what?
STEPHANIE: [laughs]
JOËL: Or maybe I kind of understand the idea of, like, okay, it's a way to, like, be able to step through a collection. But thinking in terms of a cursor or even your model as a cassette tape, I think that gives me a model, not just for enumerators, but then for better understanding that external iterator pattern. Like, I'm now not going to think of if I'm ever reading through the Gang Of Four book, or some other languages say we're an doing External Iterator Pattern, and I'll immediately be like, oh, that's a cursor, or that's a cassette tape.
STEPHANIE: Yeah, very cool. I like it.
JOËL: Another mental model that I have is thinking of enumerator in terms of a lazy collection. This is something that you tend to see more in functional programming languages, so the idea that you have a collection of potentially infinite length, or it could even be unknown length. But each element only sort of comes into being as you attempt to read it. So, it's kind of, like, a potentially infinite chain of Schrodinger's boxes. And you've got to open each of them to find out what's inside.
STEPHANIE: Do you know what this reminded me of? Like elementary school math questions that were like, "What comes next in this pattern?" And it has, like, you know, the first, like, four or five values in a sequence or something. And then, you have to figure out, like, what the next value is. But then, in some ways, you know, I think it can depend on whether your enumerator is using the previous value to determine the next one. But yeah, it's like, you can't just jump ahead to figure out what the 10th, you know, value in this pattern is without kind of knowing what's come before it.
JOËL: And sort of that needing to step through the entire collection, sort of one element at a time.
STEPHANIE: Yeah, exactly.
JOËL: I think a way that that concept is interesting, to me, is situations where a collection might be expensive, and you don't necessarily need all of it. So, you might have a bunch of calculations, but you can stop when you've hit the first one that succeeds or that matches a certain criteria. And so, it's not worth it to calculate the entire array of calculations if you're going to stop at the third one. And you could do that with some sort of, like, loop or something like that. But having it as a collection means you get to just treat it like an array, and you can call detect on it and do all the nice things that you're used to. It just happens to be a little bit more efficient in terms of not creating more data than you need to.
STEPHANIE: Yeah. And I think there's some really cool stuff you can do when you start chaining enumerators with this concept of it being lazy evaluated. So, one of the things I learned in my deep dive is that when you are using the lazy method, you're able to chain enumerators. And they work a bit differently, where the default functionality is, like, everything in the collection gets evaluated through the first method, and then it gets iterated over in the second method. Whereas if you use lazy, I believe how it works is that, like, the first value gets kind of processed by all of the methods. And then, you get, you know, the output before moving on to the second, like, the next value. Does that sound right?
JOËL: Yes. And I think that's where there's often a lot of confusion because there's sort of plain enumerator, and then there's a lazy enumerator that Ruby provides. A plain enumerator is a lazy list in the sense that items don't get evaluated unless you try to reach for them. So, if you have an enumerator and you say, "Just give me the first five items," it will do that. And even if the collection was 200 items long, the next 195 don't get evaluated. So, that's very efficient there.
Where you would get into trouble is that plain enumerators are not lazy when it comes to traversals. So, any method that would traverse the entire collection, so something like a map or a select, is not going to be lazy because it's going to traverse the entire collection, therefore forcing us to evaluate each of the items in there. Whereas something like enumerable lazy will not actually traverse the collection when you do your map or you're selecting. It will wait for you to say, "Give me the first item," or "Give me the first ten items," or something like that. But you don't always need lazy. You really only need lazy when you're doing a traversal method.
STEPHANIE: Okay. Cool, cool, cool. That makes a lot of sense.
JOËL: I think a sort of spinoff metaphor that I have there is this idea of a lazy list. Another concept that, in my mind, is very adjacent to lazy lists is the concept of streams. And streams I typically think of them in terms of, like, files or networking, things like that. But a thing that you can do let's say you're working on data that's in a very large file, so big that you can't fit it into memory, a common solution there is streaming it. So, you don't load the entire file into memory and then operate on it. Instead, little chunks of it are loaded into memory. You operate on them, and then you release that memory and load the next chunk. So, you sort of work through that file in chunks, but you'd only have, you know, 1 line or ten lines or however big your chunk is in memory at a time.
An enumerator allows you to do that with things that are not files. So, this could be a situation where, let's say, you're reading a lot of data from the database. You just have too many rows. You can't load them all into memory at once. But you do want to traverse through them. You could chunk that using enumerator so that every, you know, it loads 100 rows at a time or 1,000 rows at a time, or something like that. And your enumerator allows you to treat that as though it's a single array, even though, in the background, it's being chunked into pieces so that you never have more than a thousand rows at a time in memory. So, it allows you to do some, like, really nice sort of memory performance things.
STEPHANIE: When would you want to use this over kind of something like batching queries?
JOËL: So, I think ActiveRecord find_in_batches does something like this under the hood.
STEPHANIE: Oh, cool.
JOËL: I don't know if they use Ruby's enumerator or if they sort of build their own custom extension to it, but it's built on this idea.
STEPHANIE: Okay, that's really neat. I have another mental model that I wanted to get your thoughts on.
JOËL: Yeah!
STEPHANIE: One of the ways that I looked up that you can construct an enumerator, an infinite enumerator like we were talking about a little bit earlier, was with the produce class method. And that actually got me thinking about a production line and this idea that, you know, you have this mechanism for, you know, producing some kind of material or, like, good or something like that. And it's just there and waiting and ready [laughs] for you to, like, kind of ask for it, like, what it needs to do. And you can do that, like, sometimes in batches, right? If you are asking for like, "Okay, I want a thousand units," and then the production line goes to work [laughs]. But yeah, that was another one of those things where I'm like, wow, they really, I think, came up with a cool method name that evoked, like, an image in my head.
JOËL: That's the power of naming, right? And I think it's interesting you've mentioned twice how going through the method names on enumerator and finding different method names all of a sudden, like, turned on a light bulb in your mind. So, if you're naming things well, it can be incredibly useful for users of your library to pick up on what you're trying to do.
So, I want to circle back to something that you mentioned earlier, the idea of elementary school quizzes where you have to, like, figure out the next item in the sequence. Because that, for me, is very similar to my mental model: the idea that an enumerator is a sequence generator. So, instead of thinking of it as, oh, it's like an array or it's some kind of collection, instead, think of it as a robot that I can just ask it, hey, give me a value, and it will give me a value. And then, it will, like, keep doing that as long as I keep asking it for it. And those values, you know, they could be totally random. You can build one of those.
But you can also have it so that the values sort of come from a sequence. It's not like an array where you're like, oh, I'm going to, like, predefine an array of, I don't know, the Fibonacci sequence, and when someone asks me for the third value, I'll just go and read that third value from the array. Instead, it knows the algorithm, and it just says, "Oh, you want the next value in the Fibonacci sequence? Let me calculate it. Here it is. Oh, you want the next value? Here it is." And so, thinking from that perspective helped me really come to terms with the concept that values really do get calculated just in time. It's not really a collection. It's an object that can give you new values if you ask it.
STEPHANIE: Yeah, okay. That is making a lot more sense kind of in conjunction with the lazy list model that you shared earlier, and even a little bit with the production line that I was kind of sharing where it's like, you know, in this case, kind of, it's, like, the potential for a value, right?
JOËL: Right, exactly. And, you know, these are all mental models that converge on the same ideas because they're all just slightly different perspectives on what the same object does. And so, there is going to be some overlap, some converging between all of them. I have another fun one. Can I throw it at you?
STEPHANIE: Please.
JOËL: This one's a little bit different, and it's the idea that enumerators are a tool to bring your own iteration to a collection. So, imagine a situation where you're building your own, let's say, binary tree implementation. And there are multiple ways to traverse through a binary tree. In particular, let's say you're doing depth-first search. There are sort of three classic ways to traverse that are called pre-order, post-order, and in-order traversals. And it really is just sort of what order do you visit all the children in your tree?
Now, the point of a collection, oftentimes, is you need a way to iterate through it. And a classic solution would be to include enumerable, the module. In order to do that, you have to define a way to iterate through your collection. You call that each. And then, enumerable just gives you all the other nice things for free. The question is, though, for something like a tree where there are multiple valid ways to traverse, which one do you pick to make it the each that gets sort of all the enumerable goodies, and then the others are just, like, random methods you've defined?
Because if you define, let's say, pre-order traversal as each, now your detect and select and all those are going to work in pre-order, but the others are not going to get that. So, if you map over a tree, you're forced to map over in pre-order because that's what the library author chose. But what if you want to map over a tree in post-order or in-order?
STEPHANIE: Yeah, well, I'm guessing that here's where enumerator comes in handy [laughs].
JOËL: Yes. The approach here is instead of designating sort of one of those traversals as the sort of blessed traversal that gets to have enumerable; you build three of these, one for each of these traversals. And then, what's really nice is that because enumerators are themselves enumerable, they have map and select and all of these things built in.
Now you can do something like mytree dot preorder dot map or mytree dot postorder dot map. And you get all the goodies for free, but the users of your library get to basically choose which traversal they want to have. As a library author, you're not forced to pick ahead of time and sort of choose; this is the one I'm going to have. You sort of bring your own traversal by providing an enumerator, and then everything else just kind of falls into place.
STEPHANIE: Bring Your Own Traversal (BYOT) [laughter]. I like it. Yeah, that's cool. I can see how that would be really handy. I have not yet encountered a situation where I needed to get that deep into how my iteration is traversed, but that's really interesting. And, I mean, I can start even imagining, like, having an each method defined in these different ways, and then all of that being able to be composed with some of the other...just other methods. And now you have, like, so many different ways to perhaps, like, help, you know, different performance use cases.
JOËL: Yeah, it can be performance. I often tend to think of enumerator as a performance thing because of its sort of lazy properties because; it allows you to sort of stream or chunk data that you're working with. But in the case of this mental model of the Bring Your Own Traversal, it actually is more about flexibility and having sort of the beauty of Ruby without having to compromise on, oh, I have to pick a single way to traverse a collection.
STEPHANIE: But I really appreciate kind of this discussion about enumerator because this was previously, like, I don't think I have really ever used the class itself to solve a problem, but now I feel a lot more equipped to do so with a couple of the different kind of perspectives. And I think what they helped me do is just prime myself. If I see a problem that might benefit from something being iterated in a lazy way, like, being like, oh, I remember this thing, this mental model. Now I can go kind of look at the documentation for how to use it. And yeah, like, I don't know how I would have stumbled across, like, reaching for it otherwise.
JOËL: That's a really interesting thing to notice because we've been talking a lot about how mental models can be a tool for understanding. But once you build an understanding, even though it's somewhat fuzzy, they're also a great tool for sort of recall. So, not only are you thinking, okay, well, this mental model says enumerators are kind of like this, or they function in this way.
On the flip side of it, you can say, "Well, lazy evaluation problems are often enumerator problems. Like, streaming or chunked data problems are often enumerator problems. Multiple traversals are enumerator problems." So, now, even though you don't, like, fully understand it in your mind, you've got that recall where you can enter it, where you can come across that problem, and immediately you're like, oh, I'm dealing with multiple traversals here. I don't remember exactly how, but somehow, in my mind, I've got a connection that says, "Enumerators are a solution for this. Let me dig into that."
STEPHANIE: Yeah, especially as an alternative to where I would normally reach for something...a more kind of common enumerable method. Because I definitely know that feeling of like, oh, like, I wish it could just, like, do this a little bit differently, you know. And it turns out that, you know, something like that probably exists already. I just needed to know what it was [laughs].
JOËL: On that theme of I wish that I could have something that behaved just a little bit more...like, I'm doing something slightly weird, and I wish they would behave more, like, just plain Ruby does normally with my, like, collections I'm familiar with.
I'm going to pitch a talk that I gave at RubyConf Mini called "Teaching Ruby to Count." Some of these mental models actually showed up there. But the whole idea is like, oh, if you're bringing in sort of more custom objects and all of that, how can you just tweak them a little bit so that they're just as joyful to use and interact with as arrays, and numbers, and ranges? And they just sort of fit into that beauty of Ruby that we get out of the box.
STEPHANIE: Awesome. On that note, shall we wrap up?
JOËL: Let's wrap up.
STEPHANIE: Show notes for this episode can be found at bikeshed.fm.
JOËL: This show has been produced and edited by Mandy Moore.
STEPHANIE: If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review in iTunes. It really helps other folks find the show.
JOËL: If you have any feedback for this or any of our other episodes, you can reach us @_bikeshed, or you can reach me @joelquen on Twitter.
STEPHANIE: Or reach both of us at hosts@bikeshed.fm via email.
JOËL: Thanks so much for listening to The Bike Shed, and we'll see you next week.
ALL: Byeeeeeeee!!!!!!!
AD:
Did you know thoughtbot has a referral program? If you introduce us to someone looking for a design or development partner, we will compensate you if they decide to work with us.
More info on our website at: tbot.io/referral. Or you can email us at: referrals@thoughtbot.com with any questions.
442 epizódok
Manage episode 422958694 series 1401614
Joël explains his note-taking system, which he uses to capture his beliefs and thoughts about software development. Stephanie recalls feedback from her recent RailsConf talk, where her confidence stemmed from deeply believing in her material despite limited rehearsal. This leads to a conversation about the value of mental models in building a comprehensive understanding of a topic, which can foster confidence and adaptability during presentations and discussions.
The episode then shifts focus to the practical application of enumerators in Ruby, exploring various mental models to understand their functionality better. Joël introduces several metaphors, such as enumerators as cursors, lazy collections, and sequence generators, which help demystify their use cases.
- Episode on note-taking
- What we believe about software
- Ruby Enumerators
- Enumerator Lazy
- Modeling a Paginated API as a lazy stream
- Solving a memory performance issue with enumerator
- Find in batches
- Binary tree implementation with different traversals
- Teaching Ruby to Count
Transcript:
STEPHANIE: Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Stephanie Minn.
JOËL: And I'm Joël Quenneville, and together, we're here to share a bit of what we've learned along the way.
STEPHANIE: So, Joël, what's new in your world?
JOËL: So, what's new in my world isn't exactly a new thing. I've talked about it on the podcast here before, and it's my note-taking system. I have a system where I try to capture notes that are things I believe about software or things I think are probably true about software. They're chunked up in really small pieces, such that every note is effectively one small thesis statement and a paragraph of text, and maybe a diagram or a code snippet to support that. And then, it's highly hyperlinked to other notes. So, I sort of build out some thoughts on software that way.
A thing that I've done recently that's been pretty exciting with that is introducing a sort of separate set of notes that connect to my sort of opinion notes. So, I create individual notes for public works that I've done, things like blog posts or conference talks. Because a lot of those are built on top of ideas that have been sitting in my note system for a while. Readers and listeners get to sort of see the final product, but often sort of built up over several months or even a couple of years as I added different notes that kind of circled a topic and then eventually got to a thing.
What I did, though, was actually making those connections explicit. And so I use Obsidian. Obsidian has this cool graph view where it just sort of shows all of the notes, and it circles them with, like, connections between them where the notes connect. So, I can now see in a visual format how my thoughts cluster in different topics, but then also which clusters have talks and blog posts hanging off of them and also which ones don't, which ones are like, oh, I have a lot of thoughts on this topic, and I've not yet written about it in a public forum; maybe that would be a thing to explore. So, seeing that visual got me really excited. I was having a good time.
STEPHANIE: Yes, I have several thoughts coming to mind in response, which is, I know you love a visual. I really like the system of, even if you have created content for it, like, you have a space for, like, thoughts about it to evolve. Because you said, like, sometimes content comes out of notes that you've been...or, like, thoughts you've been having over years, but it's like, even afterwards, I'm sure there will still be new thoughts about it, too. I always have a hard time finding a place for that thing kind of once I, I don't know, it's like some of that stuff is never really considered done, right? So, that is really cool.
And I also was just thinking about an old episode of The Bike Shed back when Chris Toomey and Steph Viccari hosted the podcast called "What We Believe About Software," I think, is the title. And I was just thinking about how, like, if only we could just dump all of your notes [laughs] into some, you know, stream [laughs], and that would be really cool. If we ever do, like, an episode like that, that would be really fun. And I'm sure, you know, you already have this, like, huge bank of ideas [laughs].
JOËL: Yes. It is really fun because I build up...the thoughts are often sort of interconnected, and so they might have a topic, but they are very focused. So, I might have, like, three or four things I believe about a particular topic that cluster together. So, we could...and, actually, I have used, in the past, some of those clusters as initial food for thought for a Bikeshed episode.
STEPHANIE: Yeah, that's really neat. I like this idea of a kind of just, like, a repository for putting down what you believe about software as kind of, like, guiding principles for yourself as a developer a little bit.
I remember a piece of feedback I got about my RailsConf talk that I gave a few weeks ago, and someone said like, "Oh, you sounded really confident in what you were talking about." And that surprised me because I, like, didn't practice rehearsing giving the talk all that much [laughs]. It's because they had asked like, "Oh, like, did you practice a lot?" or something like that. And I think I realized that I, like, really believed in what I was sharing and kind of that, I think, was perhaps what they were picking up on.
And even though, like, maybe the rehearsal of the presentation itself was not where I had spent a lot of time on, I had spent a lot of time thinking about what I wanted to share and just building up my confidence around that. So, I thought that was an interesting connection.
JOËL: Yeah, you fully developed the idea. You kind of explored all the side trails, maybe a little bit on your own as well. You're on very familiar terrain. And so, that is a way of building confidence separate from just sort of memorizing a talk.
STEPHANIE: Yeah, yeah. Exactly.
JOËL: In a sense, I almost feel like that's a better sense of confidence because then you can sort of...you can roll with the punches. You know, if a slide is out of order or something, sure, it maybe messes up a little bit of the narrative that you're trying to say. But you're not like, "Oh no, what is this content?" You're like, "Oh yeah, this thing," and you can dive right into it. Somebody asks you a question, and you're not like, "Oh no, that was not in the script," because, again, you've sort of mastered your topic. You know the area as a whole, even sort of the blurry edges beyond the talk, and can react in a way that is pretty confident.
STEPHANIE: Yeah. I still definitely fear the open Q&A. I've never done it before, but maybe one day I will be able to because I just, you know, know my topic so well inside and out [laughs] that I can roll with the punches, as you say.
JOËL: Open Q&A is just...it's a roll of the dice. Sometimes, you get some really good conversation topics there, and sometimes, it's just a waste of everyone's time.
STEPHANIE: I like that take [laughs].
JOËL: Maybe that should go into the things I believe about software. So, other than receiving feedback about your RailsConf talk, what is new in your world?
STEPHANIE: Yeah, so I am wrapping up a pretty large project on my client work that we're hoping to release soon. And, in fact, it's actually being released along with a big announcement from the client company to their customers. Essentially, at a conference, they're going to say like, "Hey, like, we now have this new feature." And so, I think there's some hype generated around it. And this past week, we've been doing a lot of internal testing of the feature because there are a lot of employees of my client company who are, like, pretty big users of the product, which is cool because I think we're getting, you know, we have easy access to people who can give us good feedback.
But I am having a hard time with being on the receiving end of the feedback and figuring out, like, what is stuff I need to attend to now before, you know, this big release? And what is stuff that is just kind of, like, general feedback like, "Oh, like, I wish it did this," but, you know, it turns out that that's not really what we were building? And how do I just kind of, like, accept that?
You know, it's coming from a good place, but I can't really help them there, at least right now. And that's hard for me because I like helping people, right? And so, if someone says something like, "Oh, like, I wish it did this," or like, "Oh, that's kind of weird," I'm like, "Oh, I want to just, like, fix that for you right now [laughs]." And I suspect that a lot of other devs can relate to this, especially if, like, you know, you've been working on something for a little bit, and it feels...I'm just going to say it: it feels a little precious to me.
So, what I'm trying to do today, actually, is not look at any of the feedback at all [laughs] and come at it tomorrow with a bit of a calmer vibe and be able to separate out, like, you know, I think all feedback is informative, but not all of it is useful for you at any given moment. Like, if there are bugs, then those will be my immediate priority. If there's maybe some small tweaks that we can make the feature just a little bit more polished, then I also think those are good.
But then we are discovering a few things, too, about, like, what this feature is or could be. And I think those are the things that, you know, need to be brought into a conversation with a broader group and think about, like, is this the direction we want to go? So, that's kind of how I'm bucketing that feedback right now.
JOËL: How do you feel about receiving direct feedback versus having something filtered through something like a product team?
STEPHANIE: Ooh, that's an interesting question. Because right now we're doing, I think, a mix of both that I'm not sure that I really like. On one hand, when it's filtered, it's hard to get to the root of what someone is asking for. And oftentimes, like, it may not even include enough information after the fact to be able to come at it from a dev perspective. But then direct feedback, I think, is just a little bit overwhelming sometimes. And it can be hard to figure out what to pay attention to if you don't have that, like, input from a product team about, like, what the roadmap is looking like or where, you know, strategically their heads are at.
So, one thing that kind of has emerged from this is like, oh, I was getting, you know, notifications for the feedback coming in. And what we did was set up a meeting [laughs] so that we can...maybe all of us can, like, scan it together ahead of time and then come at it with a little bit of context about what's come in but then maybe coalesce around the things that we feel are important.
JOËL: Well, you'll have to keep us updated on how that plays out, and we can kind of hear what is the balance that ends up working well for you.
STEPHANIE: Yeah, I hope so. I think this is actually maybe something that's a bit underexplored from the dev perspective, you know, that in-between stage of you're not totally done because it's not shipped to the world yet, but, you know, you're starting to get a little bit of that input. And what you do with that? Because I think there is some value in being engaged in that process.
JOËL: So, we were talking earlier about this note-taking system that I use and sort of a renewed excitement that I have about it. And one thing that I did when I was going through and finding clusters of things that hadn't been written about was I found that I had a cluster of notes on different mental models that I had for understanding Ruby enumerators, not the enumerable module, but the enumerator object. And I decided, you know what? This would probably make for a good blog post. So, I drafted a blog post, and I've been thinking about this a little bit more recently. So, I've been really hyped about digging into enumerators because of that experience.
STEPHANIE: Yeah, that's very cool. I have to say that I feel like I did not know a lot about enumerators and the API for them kind of before you brought this topic up, and I did a bit of a deep dive in preparation for us to discuss it. I feel like most devs, you know, work with enumerators via methods on enumerable without totally knowing that they are. So, I think that this would be a really interesting episode for people to be like, oh, like, I've been using this stuff, you know, the whole time, and now I can have a different perspective or just more insight on what they can do.
JOËL: Before we dig into individual mental models, though, I want to think a little bit about the concept of mental models as a whole. Years ago, someone gave me advice to sort of pay attention to mental models, ways I think about the world or different code structures, different code approaches, and that really stuck with me. So, I've since been, like, kind of, like, collecting mental models.
And, in a way, they're like a, for me, a bit more of a concrete way to look at a particular topic. So, I can say I'm looking at this particular topic through the lens of a particular mental model that helps me build more clarity around it. And if I have three or four, then I can kind of look at it from three or four different perspectives. And now, all of a sudden, I feel like I'm seeing in three dimensions.
STEPHANIE: Whoa, the Matrix even [laughs]. That's cool. Yeah, I really like that advice. I think I'm going to steal it and start kind of suggesting it to other people because I think, in a way, on this show, that has come through a lot. And talking about things on the podcast has helped me develop a lot of my mental models. And I think we've done a few, like, episodes in the past about various ones we have for just our work because it's like, that's infinite [laughs]. But what I really have been appreciating is that mental models just need to work for you. As long as you're able to understand something, then it's valuable.
And that has really helped me also, like, just get on the same understanding with others because the goal is not necessarily to, like, explain it the way that I would think of it, but figure out what would help them kind of develop their own mental model for understanding something, and, you know, kind of as long as we both feel like we have that shared understanding, no matter what lens it's through. And, you know, sometimes it's even more effective when we are able to share it. But I feel like, you know, you can still find ways to collaborate on something with a diversity of mental models.
JOËL: Yeah, they're a great way to build self-understanding. They're a great way to sort of build understanding between two people. So, I'm a huge fan of the concept. And part of what I've been doing with my note-taking system is trying to capture those as much as possible. If I'm ever, like, trying to understand a complex topic and I'm like, oh, I think I've got a breakthrough here; I understand it; it's kind of like this, or you can imagine it in this perspective, it's like, write that down. That's gold.
STEPHANIE: Very cool. So, Joël, would you be able to share some of your mental models for enumerator?
JOËL: So, one way that I look at it is the idea that an enumerator is effectively a cursor over a collection. So, you have an array and a regular array; you're either in the middle of iterating through it using something like each, or you're not. You just have a collection of items. Enumerator introduces the idea that you're actually sort of at a position in the array. So, you're sort of focused on, let's say, the third item or the fourth item. You have a cursor there, and you can move that cursor forward as you sort of step through.
But the really cool thing is you can also kind of pause and just pass that cursor on to someone else, and someone else can move the cursor a few steps further down the collection, pause, pass it on to someone else. And it's totally fine. Nobody has to, like, go through an entire, like, each iteration.
STEPHANIE: Yeah. So, when you were talking about cursors, that got me thinking a little bit because I actually have struggled with that concept, especially when it comes to, you know, things code-related. Like, when I've had to work with database things and stuff, like, the idea of a cursor was a little, like, difficult for me to wrap my head around. And I was looking at the methods on enumerator, like the instance methods on enumerator. And one of them actually is what helped me develop this mental model. And I'm excited to see what you think.
But there is a rewind method that basically rewinds the sequence back to its beginning, right? And what that triggered for me was a VHS tape [laughs] and just those, like, car-shaped rewinders for tapes back in the '90s. I don't know if you ever had one in your house, but I did. And I just thought that was such a cool method name because it was very, I don't know, it was just like a word that we use in the English language, right?
So, the idea of, like, tapes, you know, like, cassette tapes or VHS tape kind of also it sounds like it matches well with what you were sharing, too, where it's like, I could pass, I don't know, maybe I, like, listen to a few songs on my cassette tape, and then I give it to someone else, and they can pick up where I left off. And yeah, that was really helpful in understanding, like, a marker of a position a little more than cursor was able to for me.
JOËL: That's really interesting because now I wonder, like, how far we could push that metaphor. So, musical data is encoded on magnetic tape. Cassette tapes typically there are sort of two spools. You start off with all of the tape wound up around one spool, and then as it sort of moves across the read head, it gets wound up on sort of the, I don't know, destination spool. I guess you can call them origin and destination. And because of that, you can sort of be in a, like, partly read state where, you know, half the tape is on the destination spool, half of it is on the origin spool, and you have that read head that's in the middle, and you're just kind of paused there. And you can kind of jump forward in that.
So, I imagine something like that in your metaphor is like an enumerator. Contrast that to imagine just a single spool, which is just we have musical data encoded on magnetic tape, and we wrapped it up on a spool. I feel like that's almost more like a regular array because you don't have that concept of, like, position, or being able to read parts of it or anything like that. It's just, here's some data.
STEPHANIE: Yeah. While you were talking about the two spools, I was thinking about, like, part of what is nice about enumerator is that you can go forward or backwards, right? And that feels a little more possible with that two-spool metaphor [laughs], rather than just unraveling something, where you are kind of discarding what has already been read.
JOËL: The one caveat there is that enumerators can move forward one item at a time. They can only move backwards by jumping back to the beginning. So, you can't step forward or step back.
STEPHANIE: Yeah, that's fair.
JOËL: You step forward, or you, like, rewind to the beginning. I think, in my mind, I was thinking a little bit more about this metaphor. And I think it's also just a metaphor for what's called the External Iterator Pattern. It's one of the classic Gang of Four Patterns, which is what enumerator, the object in Ruby, is an implementation of. I feel like I always see that in the documentation, like, oh, enumerator is an implementation of the External Iterator Pattern. And I just kind of go, what?
STEPHANIE: [laughs]
JOËL: Or maybe I kind of understand the idea of, like, okay, it's a way to, like, be able to step through a collection. But thinking in terms of a cursor or even your model as a cassette tape, I think that gives me a model, not just for enumerators, but then for better understanding that external iterator pattern. Like, I'm now not going to think of if I'm ever reading through the Gang Of Four book, or some other languages say we're an doing External Iterator Pattern, and I'll immediately be like, oh, that's a cursor, or that's a cassette tape.
STEPHANIE: Yeah, very cool. I like it.
JOËL: Another mental model that I have is thinking of enumerator in terms of a lazy collection. This is something that you tend to see more in functional programming languages, so the idea that you have a collection of potentially infinite length, or it could even be unknown length. But each element only sort of comes into being as you attempt to read it. So, it's kind of, like, a potentially infinite chain of Schrodinger's boxes. And you've got to open each of them to find out what's inside.
STEPHANIE: Do you know what this reminded me of? Like elementary school math questions that were like, "What comes next in this pattern?" And it has, like, you know, the first, like, four or five values in a sequence or something. And then, you have to figure out, like, what the next value is. But then, in some ways, you know, I think it can depend on whether your enumerator is using the previous value to determine the next one. But yeah, it's like, you can't just jump ahead to figure out what the 10th, you know, value in this pattern is without kind of knowing what's come before it.
JOËL: And sort of that needing to step through the entire collection, sort of one element at a time.
STEPHANIE: Yeah, exactly.
JOËL: I think a way that that concept is interesting, to me, is situations where a collection might be expensive, and you don't necessarily need all of it. So, you might have a bunch of calculations, but you can stop when you've hit the first one that succeeds or that matches a certain criteria. And so, it's not worth it to calculate the entire array of calculations if you're going to stop at the third one. And you could do that with some sort of, like, loop or something like that. But having it as a collection means you get to just treat it like an array, and you can call detect on it and do all the nice things that you're used to. It just happens to be a little bit more efficient in terms of not creating more data than you need to.
STEPHANIE: Yeah. And I think there's some really cool stuff you can do when you start chaining enumerators with this concept of it being lazy evaluated. So, one of the things I learned in my deep dive is that when you are using the lazy method, you're able to chain enumerators. And they work a bit differently, where the default functionality is, like, everything in the collection gets evaluated through the first method, and then it gets iterated over in the second method. Whereas if you use lazy, I believe how it works is that, like, the first value gets kind of processed by all of the methods. And then, you get, you know, the output before moving on to the second, like, the next value. Does that sound right?
JOËL: Yes. And I think that's where there's often a lot of confusion because there's sort of plain enumerator, and then there's a lazy enumerator that Ruby provides. A plain enumerator is a lazy list in the sense that items don't get evaluated unless you try to reach for them. So, if you have an enumerator and you say, "Just give me the first five items," it will do that. And even if the collection was 200 items long, the next 195 don't get evaluated. So, that's very efficient there.
Where you would get into trouble is that plain enumerators are not lazy when it comes to traversals. So, any method that would traverse the entire collection, so something like a map or a select, is not going to be lazy because it's going to traverse the entire collection, therefore forcing us to evaluate each of the items in there. Whereas something like enumerable lazy will not actually traverse the collection when you do your map or you're selecting. It will wait for you to say, "Give me the first item," or "Give me the first ten items," or something like that. But you don't always need lazy. You really only need lazy when you're doing a traversal method.
STEPHANIE: Okay. Cool, cool, cool. That makes a lot of sense.
JOËL: I think a sort of spinoff metaphor that I have there is this idea of a lazy list. Another concept that, in my mind, is very adjacent to lazy lists is the concept of streams. And streams I typically think of them in terms of, like, files or networking, things like that. But a thing that you can do let's say you're working on data that's in a very large file, so big that you can't fit it into memory, a common solution there is streaming it. So, you don't load the entire file into memory and then operate on it. Instead, little chunks of it are loaded into memory. You operate on them, and then you release that memory and load the next chunk. So, you sort of work through that file in chunks, but you'd only have, you know, 1 line or ten lines or however big your chunk is in memory at a time.
An enumerator allows you to do that with things that are not files. So, this could be a situation where, let's say, you're reading a lot of data from the database. You just have too many rows. You can't load them all into memory at once. But you do want to traverse through them. You could chunk that using enumerator so that every, you know, it loads 100 rows at a time or 1,000 rows at a time, or something like that. And your enumerator allows you to treat that as though it's a single array, even though, in the background, it's being chunked into pieces so that you never have more than a thousand rows at a time in memory. So, it allows you to do some, like, really nice sort of memory performance things.
STEPHANIE: When would you want to use this over kind of something like batching queries?
JOËL: So, I think ActiveRecord find_in_batches does something like this under the hood.
STEPHANIE: Oh, cool.
JOËL: I don't know if they use Ruby's enumerator or if they sort of build their own custom extension to it, but it's built on this idea.
STEPHANIE: Okay, that's really neat. I have another mental model that I wanted to get your thoughts on.
JOËL: Yeah!
STEPHANIE: One of the ways that I looked up that you can construct an enumerator, an infinite enumerator like we were talking about a little bit earlier, was with the produce class method. And that actually got me thinking about a production line and this idea that, you know, you have this mechanism for, you know, producing some kind of material or, like, good or something like that. And it's just there and waiting and ready [laughs] for you to, like, kind of ask for it, like, what it needs to do. And you can do that, like, sometimes in batches, right? If you are asking for like, "Okay, I want a thousand units," and then the production line goes to work [laughs]. But yeah, that was another one of those things where I'm like, wow, they really, I think, came up with a cool method name that evoked, like, an image in my head.
JOËL: That's the power of naming, right? And I think it's interesting you've mentioned twice how going through the method names on enumerator and finding different method names all of a sudden, like, turned on a light bulb in your mind. So, if you're naming things well, it can be incredibly useful for users of your library to pick up on what you're trying to do.
So, I want to circle back to something that you mentioned earlier, the idea of elementary school quizzes where you have to, like, figure out the next item in the sequence. Because that, for me, is very similar to my mental model: the idea that an enumerator is a sequence generator. So, instead of thinking of it as, oh, it's like an array or it's some kind of collection, instead, think of it as a robot that I can just ask it, hey, give me a value, and it will give me a value. And then, it will, like, keep doing that as long as I keep asking it for it. And those values, you know, they could be totally random. You can build one of those.
But you can also have it so that the values sort of come from a sequence. It's not like an array where you're like, oh, I'm going to, like, predefine an array of, I don't know, the Fibonacci sequence, and when someone asks me for the third value, I'll just go and read that third value from the array. Instead, it knows the algorithm, and it just says, "Oh, you want the next value in the Fibonacci sequence? Let me calculate it. Here it is. Oh, you want the next value? Here it is." And so, thinking from that perspective helped me really come to terms with the concept that values really do get calculated just in time. It's not really a collection. It's an object that can give you new values if you ask it.
STEPHANIE: Yeah, okay. That is making a lot more sense kind of in conjunction with the lazy list model that you shared earlier, and even a little bit with the production line that I was kind of sharing where it's like, you know, in this case, kind of, it's, like, the potential for a value, right?
JOËL: Right, exactly. And, you know, these are all mental models that converge on the same ideas because they're all just slightly different perspectives on what the same object does. And so, there is going to be some overlap, some converging between all of them. I have another fun one. Can I throw it at you?
STEPHANIE: Please.
JOËL: This one's a little bit different, and it's the idea that enumerators are a tool to bring your own iteration to a collection. So, imagine a situation where you're building your own, let's say, binary tree implementation. And there are multiple ways to traverse through a binary tree. In particular, let's say you're doing depth-first search. There are sort of three classic ways to traverse that are called pre-order, post-order, and in-order traversals. And it really is just sort of what order do you visit all the children in your tree?
Now, the point of a collection, oftentimes, is you need a way to iterate through it. And a classic solution would be to include enumerable, the module. In order to do that, you have to define a way to iterate through your collection. You call that each. And then, enumerable just gives you all the other nice things for free. The question is, though, for something like a tree where there are multiple valid ways to traverse, which one do you pick to make it the each that gets sort of all the enumerable goodies, and then the others are just, like, random methods you've defined?
Because if you define, let's say, pre-order traversal as each, now your detect and select and all those are going to work in pre-order, but the others are not going to get that. So, if you map over a tree, you're forced to map over in pre-order because that's what the library author chose. But what if you want to map over a tree in post-order or in-order?
STEPHANIE: Yeah, well, I'm guessing that here's where enumerator comes in handy [laughs].
JOËL: Yes. The approach here is instead of designating sort of one of those traversals as the sort of blessed traversal that gets to have enumerable; you build three of these, one for each of these traversals. And then, what's really nice is that because enumerators are themselves enumerable, they have map and select and all of these things built in.
Now you can do something like mytree dot preorder dot map or mytree dot postorder dot map. And you get all the goodies for free, but the users of your library get to basically choose which traversal they want to have. As a library author, you're not forced to pick ahead of time and sort of choose; this is the one I'm going to have. You sort of bring your own traversal by providing an enumerator, and then everything else just kind of falls into place.
STEPHANIE: Bring Your Own Traversal (BYOT) [laughter]. I like it. Yeah, that's cool. I can see how that would be really handy. I have not yet encountered a situation where I needed to get that deep into how my iteration is traversed, but that's really interesting. And, I mean, I can start even imagining, like, having an each method defined in these different ways, and then all of that being able to be composed with some of the other...just other methods. And now you have, like, so many different ways to perhaps, like, help, you know, different performance use cases.
JOËL: Yeah, it can be performance. I often tend to think of enumerator as a performance thing because of its sort of lazy properties because; it allows you to sort of stream or chunk data that you're working with. But in the case of this mental model of the Bring Your Own Traversal, it actually is more about flexibility and having sort of the beauty of Ruby without having to compromise on, oh, I have to pick a single way to traverse a collection.
STEPHANIE: But I really appreciate kind of this discussion about enumerator because this was previously, like, I don't think I have really ever used the class itself to solve a problem, but now I feel a lot more equipped to do so with a couple of the different kind of perspectives. And I think what they helped me do is just prime myself. If I see a problem that might benefit from something being iterated in a lazy way, like, being like, oh, I remember this thing, this mental model. Now I can go kind of look at the documentation for how to use it. And yeah, like, I don't know how I would have stumbled across, like, reaching for it otherwise.
JOËL: That's a really interesting thing to notice because we've been talking a lot about how mental models can be a tool for understanding. But once you build an understanding, even though it's somewhat fuzzy, they're also a great tool for sort of recall. So, not only are you thinking, okay, well, this mental model says enumerators are kind of like this, or they function in this way.
On the flip side of it, you can say, "Well, lazy evaluation problems are often enumerator problems. Like, streaming or chunked data problems are often enumerator problems. Multiple traversals are enumerator problems." So, now, even though you don't, like, fully understand it in your mind, you've got that recall where you can enter it, where you can come across that problem, and immediately you're like, oh, I'm dealing with multiple traversals here. I don't remember exactly how, but somehow, in my mind, I've got a connection that says, "Enumerators are a solution for this. Let me dig into that."
STEPHANIE: Yeah, especially as an alternative to where I would normally reach for something...a more kind of common enumerable method. Because I definitely know that feeling of like, oh, like, I wish it could just, like, do this a little bit differently, you know. And it turns out that, you know, something like that probably exists already. I just needed to know what it was [laughs].
JOËL: On that theme of I wish that I could have something that behaved just a little bit more...like, I'm doing something slightly weird, and I wish they would behave more, like, just plain Ruby does normally with my, like, collections I'm familiar with.
I'm going to pitch a talk that I gave at RubyConf Mini called "Teaching Ruby to Count." Some of these mental models actually showed up there. But the whole idea is like, oh, if you're bringing in sort of more custom objects and all of that, how can you just tweak them a little bit so that they're just as joyful to use and interact with as arrays, and numbers, and ranges? And they just sort of fit into that beauty of Ruby that we get out of the box.
STEPHANIE: Awesome. On that note, shall we wrap up?
JOËL: Let's wrap up.
STEPHANIE: Show notes for this episode can be found at bikeshed.fm.
JOËL: This show has been produced and edited by Mandy Moore.
STEPHANIE: If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review in iTunes. It really helps other folks find the show.
JOËL: If you have any feedback for this or any of our other episodes, you can reach us @_bikeshed, or you can reach me @joelquen on Twitter.
STEPHANIE: Or reach both of us at hosts@bikeshed.fm via email.
JOËL: Thanks so much for listening to The Bike Shed, and we'll see you next week.
ALL: Byeeeeeeee!!!!!!!
AD:
Did you know thoughtbot has a referral program? If you introduce us to someone looking for a design or development partner, we will compensate you if they decide to work with us.
More info on our website at: tbot.io/referral. Or you can email us at: referrals@thoughtbot.com with any questions.
442 epizódok
All episodes
×Üdvözlünk a Player FM-nél!
A Player FM lejátszó az internetet böngészi a kiváló minőségű podcastok után, hogy ön élvezhesse azokat. Ez a legjobb podcast-alkalmazás, Androidon, iPhone-on és a weben is működik. Jelentkezzen be az feliratkozások szinkronizálásához az eszközök között.