Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

solarish stat following-up, supports for readdir. #4068

Merged
merged 1 commit into from
Dec 12, 2024

Conversation

devnexen
Copy link
Contributor

@devnexen devnexen commented Dec 2, 2024

No description provided.

@devnexen devnexen marked this pull request as ready for review December 2, 2024 09:49
@devnexen
Copy link
Contributor Author

devnexen commented Dec 2, 2024

@rustbot ready

@rustbot rustbot added the S-waiting-on-review Status: Waiting for a review to complete label Dec 2, 2024
// other entries than `.` and `..`.
// https://docs.oracle.com/cd/E86824_01/html/E54765/rmdir-2.html
assert_eq!(ErrorKind::AlreadyExists, remove_dir(&dir_path).unwrap_err().kind());
#[cfg(not(any(target_os = "solaris", target_os = "illumos")))]
assert_eq!(ErrorKind::DirectoryNotEmpty, remove_dir(&dir_path).unwrap_err().kind());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't duplicate the entire assert. Instead, you can do something like

let expected_err = if cfg!(any(target_os = "solaris", target_os = "illumos")) { ErrorKind::AlreadyExists } else { ErrorKind::DirectoryNotEmpty };
assert_eq!(expected_err, remove_dir(&dir_path).unwrap_err().kind());

// Solaris/Illumos `rmdir` call set errno to EEXIST if directory contains
// other entries than `.` and `..`.
// https://docs.oracle.com/cd/E86824_01/html/E54765/rmdir-2.html
assert_eq!(ErrorKind::AlreadyExists, remove_dir(&dir_path).unwrap_err().kind());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assertion seems to fail on illumos now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Locally it passes tough, dunno if this is due to native vs non native run.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Locally" meaning on an Illumos system? If you are passing through host errors, and host errors differ between OSes, then Miri will have to adjust the errors.

This likely also means that on your system, the tests will fail if you run them with --target x86_64-unknown-linux-gnu?

Copy link
Contributor Author

@devnexen devnexen Dec 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Locally" meaning on an Illumos system?

Yes.

If you are passing through host errors, and host errors differ between OSes, then Miri will have to adjust the errors.

This likely also means that on your system, the tests will fail if you run them with --target x86_64-unknown-linux-gnu?

And yes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, so we have two options:

  • we relax the tests, and accept that Miri behaves differently on different hosts since we just "pass through" errors
  • we make some attempts at error normalization

I worry that the latter will be an endless can of worms, we'll never get it "right". So how bad would it be to do the former? Does anyone really rely on the specific errors being emitted?

@rust-lang/miri what do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does anyone really rely on the specific errors being emitted?

The standard library does/has relied on specific error codes being returned by standard library functions, and has panicked or hit a debug assertion when an unexpected error code is produced.

I thought "pass through to the host" was something we explicitly avoided?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought "pass through to the host" was something we explicitly avoided?

For IO functions it is unclear what else we could practically do. We already pass through the actual file system of the host (i.e., this anyway requires disabling isolation), also passing through the errors codes doesn't make a big difference any more at that point.

The standard library does/has relied on specific error codes being returned by standard library functions, and has panicked or hit a debug assertion when an unexpected error code is produced.

Yes, but AFAIK this usually only checks that the error is some code allowed by POSIX -- whereas here we are apparently talking about POSIX allowing different codes, and different Unixes indeed using different codes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh dear, I see what's up here. Solarish is returning EEXIST for an attempt to remove a non-empty directory, because the POSIX description is "File Exists". But then std translates this error code to ErrorKind::AlreadyExists, adding the "already" which at least in this specific case, is definitely wrong. But when we display an io::Error, we punt to strerror which should only say "File exists" so we loop back to reasonable behavior.

So I'm okay with some non-portability here. Is not the first time the ErrorKind translation has been rough.

Copy link
Member

@RalfJung RalfJung Dec 5, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, so let's keep passing through the host error, and let's make the test accept both AlreadyExists and NotEmpty.

src/shims/unix/fs.rs Outdated Show resolved Hide resolved
src/shims/unix/fs.rs Outdated Show resolved Hide resolved
src/shims/unix/fs.rs Show resolved Hide resolved
src/shims/unix/fs.rs Outdated Show resolved Hide resolved
@RalfJung
Copy link
Member

RalfJung commented Dec 5, 2024

@rustbot author

@rustbot rustbot added S-waiting-on-author Status: Waiting for the PR author to address review comments and removed S-waiting-on-review Status: Waiting for a review to complete labels Dec 5, 2024
@devnexen
Copy link
Contributor Author

devnexen commented Dec 6, 2024

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Waiting for a review to complete and removed S-waiting-on-author Status: Waiting for the PR author to address review comments labels Dec 6, 2024
src/helpers.rs Outdated
@@ -324,6 +324,21 @@ pub trait EvalContextExt<'tcx>: crate::MiriInterpCxExt<'tcx> {
bug!("No field named {} in type {}", name, base.layout().ty);
}

fn try_project_field_named<P: Projectable<'tcx, Provenance>>(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please don't duplicate this code. Instead, use this function to implement project_field_named and projectable_has_field.

src/shims/unix/fs.rs Show resolved Hide resolved
src/shims/unix/fs.rs Show resolved Hide resolved
tests/pass/shims/fs.rs Outdated Show resolved Hide resolved
Comment on lines 1105 to 1107
let fields = &dirent64_layout.fields;
let fcount = (*fields).count().checked_sub(1).expect("invalid count");
let d_name_offset = (*fields).offset(fcount).bytes();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let fields = &dirent64_layout.fields;
let fcount = (*fields).count().checked_sub(1).expect("invalid count");
let d_name_offset = (*fields).offset(fcount).bytes();
let last_field = dirent64_layout.fields.count().strict_sub(1);
let d_name_offset = (*fields).offset(last_field).bytes();

@RalfJung
Copy link
Member

RalfJung commented Dec 6, 2024

@rustbot author

@rustbot rustbot added S-waiting-on-author Status: Waiting for the PR author to address review comments and removed S-waiting-on-review Status: Waiting for a review to complete labels Dec 6, 2024
@devnexen
Copy link
Contributor Author

devnexen commented Dec 6, 2024

I think I know why there is weird commit hours from me, if I let the VM/host hibernate for so many hours, the time from VM does not catch up once I wake it up.

@devnexen devnexen changed the title solarish stat following-up, supports for readdir/readdir64. solarish stat following-up, supports for readdir. Dec 6, 2024
@devnexen
Copy link
Contributor Author

devnexen commented Dec 6, 2024

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Waiting for a review to complete and removed S-waiting-on-author Status: Waiting for the PR author to address review comments labels Dec 6, 2024
src/shims/unix/fs.rs Outdated Show resolved Hide resolved
src/shims/unix/fs.rs Outdated Show resolved Hide resolved
@devnexen devnexen force-pushed the solarish_stat2 branch 2 times, most recently from cdfb85d to 5fbc8f9 Compare December 7, 2024 18:29
@devnexen
Copy link
Contributor Author

devnexen commented Dec 7, 2024

@RalfJung before I squash do you like it better now ?

@RalfJung
Copy link
Member

My proposal was to add a type_name: &str argument to the linux_solarish_readdir64 function.

@RalfJung
Copy link
Member

oops seems like I never actually posted that proposal here. Sorry for that! I was sure I submitted this, but apparently I replaced it by the question about readdir64 on Solarish.

@devnexen
Copy link
Contributor Author

oops seems like I never actually posted that proposal here. Sorry for that! I was sure I submitted this, but apparently I replaced it by the question about readdir64 on Solarish.

no worries, updating :)

@RalfJung
Copy link
Member

Btw, please avoid rebasing over the latest master branch when there are no conflicts. Github is terrible at dealing with force pushes during review, so the more you can avoid force pushes outside of squashing, the better. :)

src/helpers.rs Outdated
Comment on lines 345 to 348
if let Some(_f) = self.try_project_field_named(base, name).unwrap() {
return true;
}
false
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if let Some(_f) = self.try_project_field_named(base, name).unwrap() {
return true;
}
false
self.try_project_field_named(base, name).unwrap().is_some()

@RalfJung
Copy link
Member

Apart from one final nit, this looks great, thanks! Please squash the commits, then we can land this. Please use the --keep-base flag when squashing so that the force-push diff is easier to review. Then write @rustbot ready after you pushed the squashed PR.

@rustbot author

@rustbot rustbot added S-waiting-on-author Status: Waiting for the PR author to address review comments and removed S-waiting-on-review Status: Waiting for a review to complete labels Dec 12, 2024
@devnexen
Copy link
Contributor Author

devnexen commented Dec 12, 2024

Btw, please avoid rebasing over the latest master branch when there are no conflicts. Github is terrible at dealing with force pushes during review, so the more you can avoid force pushes outside of squashing, the better. :)

ok will try to remember that.

@devnexen
Copy link
Contributor Author

@rustbot ready

@rustbot rustbot added S-waiting-on-review Status: Waiting for a review to complete and removed S-waiting-on-author Status: Waiting for the PR author to address review comments labels Dec 12, 2024
@RalfJung RalfJung added this pull request to the merge queue Dec 12, 2024
Merged via the queue into rust-lang:master with commit a373764 Dec 12, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Waiting for a review to complete
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants