Adding Symmetry Between shared_ptr and weak_ptr

ISO/IEC JTC1 SC22 WG21 N???? - 2015-01-05

Arthur O'Dwyer, arthur.j.odwyer@gmail.com

Introduction
Problem 1: Fine control over weak_ptr ownership
Problem 2: Generic programming and weak_ptr
Solutions
Possible objections
    Could unlock be confused with reset?
    Could weak_ptr::unlock() be a free function instead?
    Should weak_ptr also have certain move-constructors?
Wording
    20.8.2.2 Class template shared_ptr [util.smartptr.shared]
    20.8.2.2.5 shared_ptr observers [util.smartptr.shared.obs]
    20.8.2.3 Class template weak_ptr [util.smartptr.weak]
    20.8.2.3.1 weak_ptr constructors [util.smartptr.weak.const]
References

Introduction

This paper identifies two minor inconveniences in the design of shared_ptr and weak_ptr, diagnoses them as owing to unnecessary asymmetry between those two classes, and proposes wording to eliminate the asymmetry (and thus the inconveniences).

Problem 1: Fine control over weak_ptr ownership

Consider the following code sample:

    shared_ptr<int> first_elt_strong(const shared_ptr<vector<int>>& vec) {
        return shared_ptr<int>(vec, &vec->at(0));
    }

The first_elt_strong function returns a "strong reference" (a shared_ptr) to the first element of the vector passed in as *vec. Naturally, we want the returned object (call it r) to share ownership with vec, so that the controlled vector doesn't get deallocated before the last user of r. Fortunately, there is a standard solution to this problem: shared_ptr has a two-argument constructor that allows us to set the ownership of the constructed object separately from the stored pointer. This functionality was added to the standard in proposal N2351 (July 2007) [N2351]. That proposal presented the following rationale:

Advanced users often require the ability to create a shared_ptr instance p that shares ownership with another (master) shared_ptr q but points to an object that is not a base of *q. *p may be a member or an element of *q, for example. This section proposes an additional constructor that can be used for this purpose.

That rationale applies in full force to this proposal, if you replace "shared_ptr" throughout with "weak_ptr".

The equivalent C++14-compliant code for weak_ptr looks deceptively similar on the surface:
    weak_ptr<int> first_elt_weak(const shared_ptr<vector<int>>& vec) {
        return shared_ptr<int>(vec, &vec->at(0));
    }
However, the return statement hides a non-trivial amount of code. This code first constructs a shared_ptr (incrementing the strong reference count of the controlled object), then uses that shared_ptr to initialize a return value of type weak_ptr (incrementing the weak reference count), then destroys the shared_ptr (decrementing the strong reference count): three atomic accesses, when just one would suffice.

Problem 2: Generic programming and weak_ptr

Given an arbitrary shared_ptr, write code to "weaken" the object into a weak_ptr with the same shared ownership and stored pointer.

    template<class ObjectType>
    void register_observers(ObjectType& obj) {
        auto sptr = obj.get_shared_ptr(); // for example, via shared_from_this
        auto wptr = weak_ptr<ObjectType>(sptr);
        sptr.reset(); // drop the strong reference as soon as possible
        register_observer_1(wptr);
        register_observer_2(wptr);
    }

Unfortunately, this code is not perfectly generic: it will fail (or at least do the wrong thing) in the case that obj.get_shared_ptr() returns something other than shared_ptr<ObjectType>. For example, it might return shared_ptr<BaseClassOfObjectType>. Or, in an advanced scenario, we might want to be able to "drop in" a replacement class custom_shared_ptr instead of the standard shared_ptr, in which case we would also have to replace weak_ptr in the above code with custom_weak_ptr. Notice that the only place in the code sample where an explicit type is used, is in the line concerned with "weakening" sptr.

"Weakening" a shared_ptr into a weak_ptr is not an operation that ought to force explicit types into otherwise generic code.

Solutions

We propose adding two new functionalities to the standard library.

First, weak_ptr gains a two-argument constructor to allow direct construction of weak_ptr objects with a specific shared ownership and a specific stored pointer. This constructor is exactly symmetrical with the existing two-argument constructor of shared_ptr, and allows us to rewrite our first problematic code sample (using only one atomic access) as:

    weak_ptr<int> first_elt_weak(const shared_ptr<vector<int>>& vec) {
        return weak_ptr<int>(vec, &vec->at(0));
    }

Second, for creating a weak_ptr<T> directly from a shared_ptr<T> in the context of generic programming where the type T may not be easily accessible, shared_ptr gains a const member function shared_ptr<T>::unlock(). This member function is exactly symmetrical with the existing functionality for creating a shared_ptr<T> directly from a weak_ptr<T>, which is known as weak_ptr<T>::lock(). It allows us to rewrite our second problematic code sample (in generic style, naming no types explicitly) as:

    template<class ObjectType>
    void register_observers(ObjectType& obj) {
        auto sptr = obj.get_shared_ptr(); // for example, via shared_from_this
        auto wptr = sptr.unlock();
        sptr.reset(); // drop the strong reference as soon as possible
        register_observer_1(wptr);
        register_observer_2(wptr);
    }

Possible objections

Could unlock be confused with reset?

The following code has no bugs:

    void process() {
        auto sptr = get_shared_ptr_from_somewhere();
        // let's hang onto it while doing some work
        do_work();
        // now it's safe to drop the strong reference and possibly destroy the state
        sptr.reset();
        // at this point the state will have been destroyed if there are no other references to it
        do_other_work();
    }
However, if a careless coder were to accidentally type sptr.unlock() (by analogy with mutex::unlock()) instead of sptr.reset(), the code would fail to drop the strong reference and break the invariants that might be assumed by do_other_work().

That is, while the non-static member functions mutex::lock() and mutex::unlock() are mutators, the non-static member functions weak_ptr::lock() and shared_ptr::unlock() are observers: they do not modify *this, but rather return brand-new objects and leave the old *this untouched.

If it were not too late to rename lock, we would suggest pairs such as wptr.strengthed() / sptr.weakened() or wptr.as_shared() / sptr.as_weak(). But, as it is too late, in our opinion the symmetry of wptr.lock() and sptr.unlock() is desirable enough to outweigh the advantages of any alternative naming scheme.

This is not a new class of problem; for example, careless coders have been observed in the wild typing vector.empty() instead of vector.clear() [Bug25509]. It might be wise of implementors to annotate both lock() and unlock() (and empty()) with implementation-specific attributes such as [[warn_unused_result]].

Could weak_ptr::unlock() be a free function instead?

Other commentators (before this proposal) have suggested code along the lines of

    template<typename T>
    std::weak_ptr<T> to_weak(const std::shared_ptr<T>& sptr)
    {
        return std::weak_ptr<T>(sptr);
    }
For example, [Hall] and [Bolas]. This seems like a reasonable and generic-friendly approach; in fact a free function (or somehow wrapping the function up into a traits class?) would be more friendly to generic programming, on the face of it.

The fundamental argument against a free function is symmetry. It would be unfortunate if the opposite of wptr.lock() were spelled to_weak(sptr). We could propose to add to_strong(wptr) at the same time... but the more symmetry we do add, the more glaringly the few exceptions will stand out! Unfortunately, the asymmetry of lock versus unlock will not go away by any route other than adding unlock to the library.

Should weak_ptr also have certain move-constructors?

shared_ptr has special member functions

    shared_ptr(shared_ptr&& r) noexcept;
    template<class Y> shared_ptr(shared_ptr<Y>&& r) noexcept;

    shared_ptr& operator=(shared_ptr&& r) noexcept;
    template<class Y> shared_ptr& operator=(shared_ptr<Y>&& r) noexcept;
In N3797 (a draft of C++14), weak_ptr has no special member functions dealing with rvalue references or move semantics. This means that in cases where move construction is called for (such as in return statements in the absence of RVO), copy construction will be employed instead. This might result in inefficiencies.

However, the Committee has already recognized this asymmetry and fixed it as LWG issue #2315 [DR2315].

Since one of the present proposal's code samples involves creating a weak reference from a strong reference and then immediately dropping the strong reference, one might reasonably ask whether this functionality should be supported via a constructor such as

    weak_ptr(shared_ptr<T>&& r) noexcept;
We believe the answer is no, because users would certainly expect this constructor to be both efficient and atomic, and implementors might find it difficult to achieve either of those goals. In the most obvious implementation, the constructor would have to simultaneously (atomically) increment a "weak reference count" and decrement a "strong reference count".

Interestingly, this code currently compiles, but with "wrong" semantics:

    void register_observers(std::shared_ptr<int> sptr) {
        std::weak_ptr<int> wptr(std::move(sptr)); // destructively convert to a weak reference?
        register_observer_1(wptr);
        register_observer_2(wptr);
    }
The instance sptr does not become empty as it would if it were "moved into" another instance of shared_ptr. Instead, the rvalue reference quietly degrades itself into a const lvalue reference, and sptr keeps its old value.

This "problem" could be "fixed" by judicious deletion of weak_ptr and shared_ptr constructors and assignment operators that take rvalue references of the other type; but that might have a significant effect on existing codebases, and is outside the scope of the present proposal regardless.

Wording

The wording in this section is relative to WG21 draft standard N3797.

20.8.2.2 Class template shared_ptr [util.smartptr.shared]

Edit paragraph 1 as follows.

// 20.8.2.2.5, observers
T* get() const noexcept;
T& operator*() const noexcept;
T* operator->() const noexcept;
long use_count() const noexcept;
bool unique() const noexcept;
explicit operator bool() const noexcept;
weak_ptr<T> unlock() const noexcept;
template<class U> bool owner_before(shared_ptr<U> const& b) const;
template<class U> bool owner_before(weak_ptr<U> const& b) const;

20.8.2.2.5 shared_ptr observers [util.smartptr.shared.obs]

Add a new paragraph between the existing paragraphs 11 and 12.

weak_ptr<T> unlock() const noexcept;

Returns: weak_ptr<T>(*this).

20.8.2.3 Class template weak_ptr [util.smartptr.weak]

Edit paragraph 1 as follows.

// 20.8.2.3.1, constructors
constexpr weak_ptr() noexcept;
template<class Y> weak_ptr(shared_ptr<Y> const& r) noexcept;
weak_ptr(weak_ptr const& r) noexcept;
template<class Y> weak_ptr(weak_ptr<Y> const& r) noexcept;
template<class Y> weak_ptr(const shared_ptr<Y>& r, T* p) noexcept;
template<class Y> weak_ptr(const weak_ptr<Y>& r, T* p) noexcept;

20.8.2.3.1 weak_ptr constructors [util.smartptr.weak.const]

Add a new paragraph after the existing paragraph 5.

template<class Y> weak_ptr(const shared_ptr<Y>& r, T* p) noexcept;

template<class Y> weak_ptr(const weak_ptr<Y>& r, T* p) noexcept;

Effects: Constructs a weak_ptr instance that stores p and shares ownership with r.

Postconditions: use_count() == r.use_count().

References

[N2351]
Peter Dimov and Beman Dawes. "Improving shared_ptr for C++0x, Revision 2" (July 2007).
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2351.htm#aliasing.
[Hall]
Brett Hall. "Automatically Made Weak" (November 2013).
https://backwardsincompatibilities.wordpress.com/2013/11/12/automatically-made-weak/
[Bolas]
Nicol Bolas, in response to Add "weak_from_this" to std::enable_shared_from_this (August 2013):
"I would prefer something less limited: a to_weak_ptr function that takes a shared_ptr<T> and returns an appropriate weak_ptr<T>." https://groups.google.com/a/isocpp.org/forum/#!msg/std-proposals/fBe0AySrnKY/a3Ww4SWwJDUJ
[Bug25509]
Paul Pluzhnikov. GCC bug report 25509, comment #28 (August 2014):
"I have just found ~30 bugs in our code, where someone wrote ... v.empty(); // v.clear() was intended!" https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25509#c28
[DR2315]
Stephan T. Lavavej. "weak_ptr should be movable". Resolved February 2014 at Issaquah.
http://www.open-std.org/jtc1/sc22/wg21/docs/lwg-defects.html#2315