7
$\begingroup$

There are various external resources, like file handles, TCP connections, and various servers, which need to be closed in some specific way before being dropped. There are a few approaches to these that I've seen:

  1. Just leave it to the programmer to properly close the resource
  2. Provide support for automatically running a close function when the struct/object representing the resource is dropped, or in GC'd languages, a finalizer that's guaranteed to run (and push library designers to do this)
  3. Disallow automatic dropping/garbage collection of some custom types, and throw a compile-time warning if this would occur (like Rust with Result), such that library designers can make running close the only way to drop the resource from the outside
  4. Provide (and require the use of) a construct like Python's with that explicitly scopes the "lifetime" of the resource, also forcing the programmer to acknowledge the resource needs to be closed before dropping it

What are the pros and cons of these approaches, or any I've missed?

$\endgroup$
14
  • $\begingroup$ Does Swift defer count? $\endgroup$
    – Bbrk24
    Commented Jul 3, 2023 at 20:01
  • $\begingroup$ @Bbrk24 I think that's more of a more convenient way to do #1, rather than a way of forcing the programmer to properly close resources $\endgroup$ Commented Jul 3, 2023 at 20:02
  • $\begingroup$ 5. Rely on the garbage collector to call a finalizer. May not work correctly in languages with non-deterministic garbage collection. $\endgroup$
    – dan04
    Commented Jul 3, 2023 at 20:23
  • $\begingroup$ Related question from Software Engineering $\endgroup$
    – dan04
    Commented Jul 3, 2023 at 20:24
  • 1
    $\begingroup$ In general none of these "force" closing files, they just are convenient ways a programmer can choose to use to more ergonomically close things that should be closed. $\endgroup$
    – mousetail
    Commented Jul 3, 2023 at 20:32

3 Answers 3

4
$\begingroup$

Swift 5.9: noncopyable types

In Swift as it stands today, all types are implicitly copyable. That means you can write code like this with an unconstrained generic:

func copy<T>(_ x: T) -> (T, T) { (x, x) }

For these types, it's impossible to know when the lifetime ends -- every semantic copy is allowed (but not required) to be a literal copy, and this rampant copying means non-reference types don't have a notion of ownership.

Swift 5.9 will add "non-copyable" or "move-only" types. Your example of a file handle was the prime example for why this should be added:

struct FileDescriptor: ~Copyable {
  private var fd: Int32

  init(fd: Int32) { self.fd = fd }

  func write(buffer: Data) {
    buffer.withUnsafeBytes { 
      write(fd, $0.baseAddress!, $0.count)
    }
  }

  deinit {
    close(fd)
  }
}

These non-copyable types can only be borrowed (aka "shared") or consumed (aka "moved"), and aren't copied all over the place. This comes with a lot of restrictions to maintain backwards compatibility, but a language that had these from the outset wouldn't need them.

Because an instance of a non-copyable type can only be borrowed or moved, it's easy to tell where ownership lies: Are you borrowing it? If not, conglaturations, you own it. If an owned instance falls out of scope, its deinit runs. If a borrowed instance falls out of scope, its deinit does not run.

However, you don't always want to close the file when it falls out of scope -- maybe you're switching to a different model of the handle, and you need the file descriptor to stay open. For cases like that, there's the discard operator:

struct FileDescriptor: ~Copyable {
  // Take ownership of the C file descriptor away from this type,
  // returning the file descriptor without closing it
  consuming func take() -> Int32 {
    let fd = self.fd
    discard self
    return fd
  }
}

Currently, the only valid way to use discard is discard self. It's equivalent to mem::forget in Rust or GC.SuppressFinalize in C#, but the fact that you can only use it on self means that users of your API can't unwittingly cause problems by leaking resources, unless they call a special method to transfer ownership.

$\endgroup$
3
$\begingroup$

unique_ptr all the way down

One way to force the closing of resources is:

  1. Avoiding copies of references;
  2. Keeping resources alive white there are references to it.

Basically std::unique_ptr. Or the somewhat equivalent scope annotations in Rust.

Opaque resources and native finalizers

The "be closed in some specific way before being dropped" reminds me of PHP's resource management. The closing mechanism of external or native resources are never exposed. Resources appear on the code surface as an opaque handle or marker class instance.

When the instance handle is dropped on normal language, an native finalizer is executed.

$\endgroup$
1
$\begingroup$

The with statement approach is bad and should not be used for new languages

The python with statement or the C# using statement work by allowing classes to implement some interface, then run some code at the end of a block whether there was an error or not. It's basically syntactic sugar for a try/finally block.

The main issue is, with can only be used if an object is created and destroyed in the same block. This means you can't safely store any closable object in a class or struct. Even returning a file from a function safely becomes impossible, creating a small gap where an error would result in the file not being properly closed.

Consider this code segment:

def open_log_file():
    file = open("log.log", "a")
    file.write("> ")
    return file

with open_log_file() as f:
    f.write("test completed")

Here, if the file.write inside the open_log_file throws an error, the file won't be closed. You could not wrap the function in a with statement because then it would be closed before the function could return. You could wrap the function in a try/except to close the file, but you need a significant amount of code to properly reconstruct and re-throw the error after closing the file.

Consider a more complex case with classes:

class MultiFile:
    def __init__(self):
        self.file_a = open("a.txt")
        self.file_b = open("b.txt")
    
    def __enter__(self):
        return self
    def __exit__(self):
        self.file_a.close()
        self.file_b.close()

with MultiFile():
    pass

Here trying to implement the context manager interface is useless if the second file fails to open during the constructor.

Of course you could mess with many nested try-except blocks to catch every possible error but if you add a few more statements it quickly becomes extremely unwieldy to handle every case properly.

Confession

I have on spent days of my life tracking down bugs caused by trying to return the value used in a with/using statement from a function. Never again.

With is also entirely superseded if you have any of the other techniques

If you close a file when dropped, with has no effect.

If you don't have exceptions, then the entire statement has no effect and the risk it tries to mitigate is non-existant.

Try/Finally is just as bad

It's also impossibly to actually catch errors at every point with this syntax. The issue is not with with specifically but anything where the main context management system limited to a single block.

It's confusing for beginners to have something that you "should always use" but doesn't work half the time

Every python or C# tutorial emphasizes that you should always use files only inside a with statement. Yet in some situations it simply doesn't work. Explaining when it wouldn't work is actually hard, and depends on a deep understanding of how the return statement and exceptions work.

If we want to teach beginners a rigorous way to ensure robust code it would be better to have a simple mnemonic to teach them that always works.

If you choose not to use with it's really hard to verify the correctness of the code.

$\endgroup$
6
  • 3
    $\begingroup$ I don't follow this argument. with is a language construct which makes it easier to manage a closeable resource when its lifetime is logically restricted to a lexical block. For uses where the resource's lifetime is not logically restricted to a lexical block, you don't use with, you call .close() manually in the right place. Removing with from the language hurts in the former case and doesn't help in the latter case. $\endgroup$
    – kaya3
    Commented Jul 3, 2023 at 21:10
  • 3
    $\begingroup$ You write "This means you can't safely store any closable object in a class or struct. Even returning a file from a function safely becomes impossible", but removing with from the language doesn't make those things safe, nor does it stop you from doing them unsafely. $\endgroup$
    – kaya3
    Commented Jul 3, 2023 at 21:12
  • 6
    $\begingroup$ There's a big difference between "with should not be the only way" and "with is bad". The vast majority of resource use fits the with model, so it's a good feature and has been provided in many languages (I first encountered it in Lisp's WITH-OPEN-FILE). $\endgroup$
    – Barmar
    Commented Jul 4, 2023 at 0:10
  • 1
    $\begingroup$ @mousetail I was mainly addressing "with can only be used if an object is created and destroyed in the same block". That's the whole point of it. $\endgroup$
    – Barmar
    Commented Jul 4, 2023 at 13:56
  • 1
    $\begingroup$ That complex uses are hard to implement correctly doesn't seem like a good objection, unless there's something better. If you try to do "multifile" yourself, you run into the same try/except hell, so why not let one implementor figure it out and package it, rather than force everyone to do it (likely getting it wrong)? $\endgroup$
    – Barmar
    Commented Jul 4, 2023 at 13:59

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .