[phobos] Making std.stdio.readf @safe

[Jakub Łabaj via phobos]

There is an idea to make stdio.readf @trusted/@safe (reported
here: https://issues.dlang.org/show_bug.cgi?id=8471). What
currently makes it unsafe is LockingTextReader using functions
FLOCK, FUNLOCK, FGETC (aliased from extern functions, dependent
on the OS) and using a cast from 'shared(_IO_FILE)*' to
'_IO_FILE*'.

I found out that stdio.write* functions are made @safe by
declaring all methods of LockingTextWriter (similar to
LockingTextReader) @trusted and using helper function:

/**
* Property used by writeln/etc. so it can infer @safe since
stdout is __gshared
*/
private @property File trustedStdout() @trusted
{
return stdout;
}

So the obvious solution is to copy the approach of stdio.write.
The other one would be to mark underlying functions
FLOCK/FUNLOCK/FGETC @trusted (which in the process would allow to
get rid off @trusted from LockingTextWriter, except casting from
shared), but I'm not sure if it's legit as there may be some
quirks and they should not be @trusted at all.

So my question are: are both solutions presented acceptable? If
yes, which one is preferred? Or maybe there is a better one?

[Walter Bright via phobos]

I haven't examined that particular issue, but @trusted applies to things with a
SAFE INTERFACE. Just sticking it on any old system function does not work. For
example, declaring strlen() to be @trusted is a giant mistake, because it's
interface is not safe.

https://issues.dlang.org/show_bug.cgi?id=8471). What currently makes it unsafe
is LockingTextReader using functions FLOCK, FUNLOCK, FGETC (aliased from extern
functions, dependent on the OS) and using a cast from 'shared(_IO_FILE)*' to
'_IO_FILE*'.
/**
*/
{
return stdout;
}
So the obvious solution is to copy the approach of stdio.write. The other one
casting from shared), but I'm not sure if it's legit as there may be some quirks
So my question are: are both solutions presented acceptable? If yes, which one
is preferred? Or maybe there is a better one?
_______________________________________________
phobos mailing list
http://lists.puremagic.com/mailman/listinfo/phobos

[Jakub Łabaj via phobos]

to things with a SAFE INTERFACE. Just sticking it on any old
system function does not work. For example, declaring strlen()
not safe.

https://issues.dlang.org/show_bug.cgi?id=8471). What currently makes it unsafe
is LockingTextReader using functions FLOCK, FUNLOCK, FGETC
(aliased from extern
functions, dependent on the OS) and using a cast from
'shared(_IO_FILE)*' to
'_IO_FILE*'.
declaring all methods
and using helper
/**
stdout is __gshared
*/
{
return stdout;
}
So the obvious solution is to copy the approach of
stdio.write. The other one
would be to mark underlying functions FLOCK/FUNLOCK/FGETC
@trusted (which in the
LockingTextWriter, except
casting from shared), but I'm not sure if it's legit as there
may be some quirks
So my question are: are both solutions presented acceptable?
If yes, which one
is preferred? Or maybe there is a better one?
_______________________________________________
phobos mailing list
http://lists.puremagic.com/mailman/listinfo/phobos

Ok, so I have doubts whether these functions can be @trusted. On
the one hand they get just FILE* as an argument which (as I see
it) makes it safe interface. On the other hand FGETC is unlocked
version of fgetc and requires explicit lock to be used safely;
FLOCK and FUNLOCK invocations should match, therefore there is
also possibility to use it incorrectly. Personally I would not
mark them @trusted then, is it correct?

[Walter Bright via phobos]

they get just FILE* as an argument which (as I see it) makes it safe interface.
On the other hand FGETC is unlocked version of fgetc and requires explicit lock
to be used safely; FLOCK and FUNLOCK invocations should match, therefore there
is also possibility to use it incorrectly. Personally I would not mark them
@trusted then, is it correct?

I'd say you're right.

[Andrei Alexandrescu via phobos]

This may be hasty. https://linux.die.net/man/2/flock does not perform
any unsafe operation, even for invalid arguments. -- Andrei

Post by Walter Bright via phobos

they get just FILE* as an argument which (as I see it) makes it safe interface.
On the other hand FGETC is unlocked version of fgetc and requires explicit lock
to be used safely; FLOCK and FUNLOCK invocations should match, therefore there
is also possibility to use it incorrectly. Personally I would not mark them
@trusted then, is it correct?

I'd say you're right.
_______________________________________________
phobos mailing list
http://lists.puremagic.com/mailman/listinfo/phobos

[Jakub Łabaj via phobos]

On Tuesday, 7 February 2017 at 17:45:00 UTC, Andrei Alexandrescu

Post by Andrei Alexandrescu via phobos
This may be hasty. https://linux.die.net/man/2/flock does not
perform any unsafe operation, even for invalid arguments. --
Andrei

Post by Walter Bright via phobos

Post by Jakub Łabaj via phobos
On the
one hand
they get just FILE* as an argument which (as I see it) makes
it safe
interface.
On the other hand FGETC is unlocked version of fgetc and
requires
explicit lock
to be used safely; FLOCK and FUNLOCK invocations should match, therefore there
is also possibility to use it incorrectly. Personally I would
not mark
them
@trusted then, is it correct?

I'd say you're right.
_______________________________________________
phobos mailing list
http://lists.puremagic.com/mailman/listinfo/phobos

FLOCK is aliased (at Linux) to
https://linux.die.net/man/3/flockfile . It may be safe on its
own, but to avoid deadlock must be followed by call to unlock.
Does it qualify to be @trusted in such case?

[Andrei Alexandrescu via phobos]

Safe/trusted means "no unsafe operation", not "will cause issues if used
incorrectly". -- Andrei

Post by Andrei Alexandrescu via phobos
This may be hasty. https://linux.die.net/man/2/flock does not perform
any unsafe operation, even for invalid arguments. -- Andrei

Post by Walter Bright via phobos

they get just FILE* as an argument which (as I see it) makes it safe interface.
On the other hand FGETC is unlocked version of fgetc and requires explicit lock
to be used safely; FLOCK and FUNLOCK invocations should match, therefore there
is also possibility to use it incorrectly. Personally I would not mark them
@trusted then, is it correct?

I'd say you're right.
_______________________________________________
phobos mailing list
http://lists.puremagic.com/mailman/listinfo/phobos

FLOCK is aliased (at Linux) to https://linux.die.net/man/3/flockfile .
It may be safe on its own, but to avoid deadlock must be followed by
_______________________________________________
phobos mailing list
http://lists.puremagic.com/mailman/listinfo/phobos

[Jakub Łabaj via phobos]

What about fgetc_unlocked then? It may be not thread safe if used
without lock, is it considered unsafe because of bad usage or
just unsafe? Sorry, I still have some problems with identifying
this.

On Tuesday, 7 February 2017 at 18:01:38 UTC, Andrei Alexandrescu

Post by Andrei Alexandrescu via phobos
Safe/trusted means "no unsafe operation", not "will cause
issues if used incorrectly". -- Andrei

Post by Jakub Łabaj via phobos
On Tuesday, 7 February 2017 at 17:45:00 UTC, Andrei

Post by Andrei Alexandrescu via phobos
This may be hasty. https://linux.die.net/man/2/flock does not perform
any unsafe operation, even for invalid arguments. -- Andrei

Post by Walter Bright via phobos

Post by Jakub Łabaj via phobos
Ok, so I have doubts whether these functions can be
@trusted. On the
one hand
they get just FILE* as an argument which (as I see it)
makes it safe
interface.
On the other hand FGETC is unlocked version of fgetc and
requires
explicit lock
to be used safely; FLOCK and FUNLOCK invocations should
match,
therefore there
is also possibility to use it incorrectly. Personally I
would not mark
them
@trusted then, is it correct?

I'd say you're right.
_______________________________________________
phobos mailing list
http://lists.puremagic.com/mailman/listinfo/phobos

FLOCK is aliased (at Linux) to
https://linux.die.net/man/3/flockfile .
It may be safe on its own, but to avoid deadlock must be
followed by
_______________________________________________
phobos mailing list
http://lists.puremagic.com/mailman/listinfo/phobos

[Jakub Łabaj via phobos]

I see it like this:
- flockfile - can be @trusted, because no matter when we call it
with correct argument, it won't do anything unsafe
- funlockfile - if called by not owning thread, the behaviour is
undefined - so potentially may do something unsafe (I don't know
what happens if called on not locked file, probably is ignored)
fgetc - when not guarded by lock it is not thread safe, shouldn't
be @trusted

Is my reasoning correct? In such case having only one of these

Post by Jakub Łabaj via phobos
What about fgetc_unlocked then? It may be not thread safe if
used without lock, is it considered unsafe because of bad usage
or just unsafe? Sorry, I still have some problems with
identifying this.
On Tuesday, 7 February 2017 at 18:01:38 UTC, Andrei

Post by Andrei Alexandrescu via phobos
Safe/trusted means "no unsafe operation", not "will cause
issues if used incorrectly". -- Andrei

Post by Jakub Łabaj via phobos
On Tuesday, 7 February 2017 at 17:45:00 UTC, Andrei

Post by Andrei Alexandrescu via phobos
This may be hasty. https://linux.die.net/man/2/flock does
not perform
any unsafe operation, even for invalid arguments. -- Andrei

Post by Walter Bright via phobos

Post by Jakub Łabaj via phobos
Ok, so I have doubts whether these functions can be
@trusted. On the
one hand
they get just FILE* as an argument which (as I see it)
makes it safe
interface.
On the other hand FGETC is unlocked version of fgetc and
requires
explicit lock
to be used safely; FLOCK and FUNLOCK invocations should
match,
therefore there
is also possibility to use it incorrectly. Personally I
would not mark
them
@trusted then, is it correct?

I'd say you're right.
_______________________________________________
phobos mailing list
http://lists.puremagic.com/mailman/listinfo/phobos

FLOCK is aliased (at Linux) to
https://linux.die.net/man/3/flockfile .
It may be safe on its own, but to avoid deadlock must be
followed by
_______________________________________________
phobos mailing list
http://lists.puremagic.com/mailman/listinfo/phobos

[Andrei Alexandrescu via phobos]

Post by Jakub Łabaj via phobos
correct argument, it won't do anything unsafe

affirmative

Post by Jakub Łabaj via phobos
- funlockfile - if called by not owning thread, the behaviour is
undefined - so potentially may do something unsafe (I don't know what
happens if called on not locked file, probably is ignored)

affirmative - in C "undefined" implies "unsafe"

Post by Jakub Łabaj via phobos
fgetc - when not guarded by lock it is not thread safe, shouldn't be
@trusted

I think you mean fgetc_unlocked? fgetc issues its own locking and unlocking.


Andrei

[Jakub Łabaj via phobos]

Ouch, yes, I meant fgetc_unlocked.

Thank you for the help! As a solution I'm going to make a PR with
the mentioned changes, i.e. copy the approach of writef and apply
@trusted to LockingTextReader - I think I can do this, because
its behaviour ensures that functions FGETC, FLOCK, FUNLOCK are
invoked in a safe manner.

On Tuesday, 7 February 2017 at 20:04:51 UTC, Andrei Alexandrescu

Post by Andrei Alexandrescu via phobos

it with
correct argument, it won't do anything unsafe

affirmative

- funlockfile - if called by not owning thread, the behaviour
is
undefined - so potentially may do something unsafe (I don't
know what
happens if called on not locked file, probably is ignored)

affirmative - in C "undefined" implies "unsafe"

fgetc - when not guarded by lock it is not thread safe,
shouldn't be
@trusted

I think you mean fgetc_unlocked? fgetc issues its own locking
and unlocking.
Andrei

[Andrei Alexandrescu via phobos]

Here some research is necessary. What I did e.g. was to google for:

is fgetc_unlocked safe?

and got a bunch of answers, til I got to
https://www.gnu.org/software/libc/manual/html_node/Character-Input.html
which in turns links to "POSIX Safety Concepts" i.e.
https://www.gnu.org/software/libc/manual/html_node/POSIX-Safety-Concepts.html.
That document sure doesn't mince words:

"MT-Unsafe, AS-Unsafe, AC-Unsafe functions are not safe to call within
the safety contexts described above. Calling them within such contexts
invokes undefined behavior."

So... the *_unlocked functions are not safe. They may be, however,
wrapped in trusted functions (that issue the appropriate locking).


Andrei

Post by Jakub Łabaj via phobos
Ouch, yes, I meant fgetc_unlocked.
Thank you for the help! As a solution I'm going to make a PR with the
to LockingTextReader - I think I can do this, because its behaviour
ensures that functions FGETC, FLOCK, FUNLOCK are invoked in a safe manner.

Post by Andrei Alexandrescu via phobos

Post by Jakub Łabaj via phobos
correct argument, it won't do anything unsafe

affirmative

Post by Jakub Łabaj via phobos
- funlockfile - if called by not owning thread, the behaviour is
undefined - so potentially may do something unsafe (I don't know what
happens if called on not locked file, probably is ignored)

affirmative - in C "undefined" implies "unsafe"

Post by Jakub Łabaj via phobos
fgetc - when not guarded by lock it is not thread safe, shouldn't be
@trusted

I think you mean fgetc_unlocked? fgetc issues its own locking and unlocking.
Andrei

_______________________________________________
phobos mailing list
http://lists.puremagic.com/mailman/listinfo/phobos

[Jakub Łabaj via phobos]

LockingTextReader does exactly so - locks in constructor, unlocks
in destructor and sometimes calls fgetc_unlocked. This makes it a
good candidate for @trusted I believe.

On Tuesday, 7 February 2017 at 21:37:49 UTC, Andrei Alexandrescu

Post by Andrei Alexandrescu via phobos
Here some research is necessary. What I did e.g. was to google
is fgetc_unlocked safe?
and got a bunch of answers, til I got to
"MT-Unsafe, AS-Unsafe, AC-Unsafe functions are not safe to call
within the safety contexts described above. Calling them within
such contexts invokes undefined behavior."
So... the *_unlocked functions are not safe. They may be,
however, wrapped in trusted functions (that issue the
appropriate locking).
Andrei

Post by Jakub Łabaj via phobos
Ouch, yes, I meant fgetc_unlocked.
Thank you for the help! As a solution I'm going to make a PR
with the
mentioned changes, i.e. copy the approach of writef and apply
@trusted
to LockingTextReader - I think I can do this, because its
behaviour
ensures that functions FGETC, FLOCK, FUNLOCK are invoked in a
safe manner.
On Tuesday, 7 February 2017 at 20:04:51 UTC, Andrei

Post by Andrei Alexandrescu via phobos

call it with
correct argument, it won't do anything unsafe

affirmative

- funlockfile - if called by not owning thread, the
behaviour is
undefined - so potentially may do something unsafe (I don't
know what
happens if called on not locked file, probably is ignored)

affirmative - in C "undefined" implies "unsafe"

fgetc - when not guarded by lock it is not thread safe,
shouldn't be
@trusted

I think you mean fgetc_unlocked? fgetc issues its own locking
and
unlocking.
Andrei

_______________________________________________
phobos mailing list
http://lists.puremagic.com/mailman/listinfo/phobos