Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Date.description in Swift #179

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

FranzBusch
Copy link
Member

Motivation

The current implementation of Date.description is depending on the what platform we are running on. On Darwin it used the CFFoundation backed implementation and on Linux/Windows it used libc APIs to do the date calculations. This caused build issues on Windows and the libc APIs are not 32bit clean.

Changes

This PR changes the implementation by using the methods implemented in swift-certificates that provide timestamp to UTC time calculations. These were added in swift-certificates for the exact same reason. Furthermore, it removes the conditional usage of CFFoundation since the output is the same.

# Result

Swift only implementation that works cross platform for Date.description which also should perform better than any libc API calls.

@FranzBusch
Copy link
Member Author

cc @compnerd This should fix your Windows issue hopefully

@FranzBusch FranzBusch changed the title Implement Date.description in pure Swift Implement Date.description in Swift Jun 14, 2023
@FranzBusch FranzBusch requested a review from parkera June 14, 2023 13:41
@iCharlesHu
Copy link
Contributor

@swift-ci please test

@compnerd
Copy link
Collaborator

Oddly enough, it does worse:

Build complete! (3.51s)
Test Suite 'Selected tests' started at 2023-06-14 20:16:52.729
Test Suite 'DateTests' started at 2023-06-14 20:16:52.729
Test Case 'DateTests.testDateComparison' started at 2023-06-14 20:16:52.729
Test Case 'DateTests.testDateComparison' passed (0.0 seconds)
Test Case 'DateTests.testDateMutation' started at 2023-06-14 20:16:52.729
Test Case 'DateTests.testDateMutation' passed (0.0 seconds)
Test Case 'DateTests.testDescription1970' started at 2023-06-14 20:16:52.729
S:\SourceCache\swift-foundation\Tests\FoundationEssentialsTests\DateTests.swift:90: error: DateTests.testDescription1970 : XCTAssertEqual failed: ("1970-01-01 00:00:00 +0000") is not equal to ("1939-1-1 0:0:0 +0000") -
Test Case 'DateTests.testDescription1970' failed (0.0 seconds)
Test Case 'DateTests.testDescriptionBeyondDistantFuture' started at 2023-06-14 20:16:52.729
Test Case 'DateTests.testDescriptionBeyondDistantFuture' passed (0.0 seconds)
Test Case 'DateTests.testDescriptionBeyondDistantPast' started at 2023-06-14 20:16:52.729
Test Case 'DateTests.testDescriptionBeyondDistantPast' passed (0.0 seconds)
Test Case 'DateTests.testDescriptionDistantFuture' started at 2023-06-14 20:16:52.729
S:\SourceCache\swift-foundation\Tests\FoundationEssentialsTests\DateTests.swift:102: error: DateTests.testDescriptionDistantFuture : XCTAssertEqual failed: ("4001-01-01 00:00:00 +0000") is not equal to ("3970-1-1 0:0:0 +0000") -
Test Case 'DateTests.testDescriptionDistantFuture' failed (0.0 seconds)
Test Case 'DateTests.testDescriptionDistantPast' started at 2023-06-14 20:16:52.729
S:\SourceCache\swift-foundation\Tests\FoundationEssentialsTests\DateTests.swift:97: error: DateTests.testDescriptionDistantPast : XCTAssertEqual failed: ("0000-12-30 00:00:00 +0000") is not equal to ("-31-12-30 0:0:0 +0000") -
Test Case 'DateTests.testDescriptionDistantPast' failed (0.0 seconds)
Test Case 'DateTests.testDescriptionReferenceDate' started at 2023-06-14 20:16:52.729
S:\SourceCache\swift-foundation\Tests\FoundationEssentialsTests\DateTests.swift:84: error: DateTests.testDescriptionReferenceDate : XCTAssertEqual failed: ("2001-01-01 00:00:00 +0000") is not equal to ("1970-1-1 0:0:0 +0000") -
Test Case 'DateTests.testDescriptionReferenceDate' failed (0.0 seconds)
Test Case 'DateTests.testDistantFuture' started at 2023-06-14 20:16:52.729
Test Case 'DateTests.testDistantFuture' passed (0.0 seconds)
Test Case 'DateTests.testDistantPast' started at 2023-06-14 20:16:52.729
Test Case 'DateTests.testDistantPast' passed (0.0 seconds)
Test Case 'DateTests.test_now' started at 2023-06-14 20:16:52.729
Test Case 'DateTests.test_now' passed (0.0 seconds)
Test Suite 'DateTests' failed at 2023-06-14 20:16:52.729
         Executed 11 tests, with 4 failures (0 unexpected) in 0.0 (0.0) seconds
Test Suite 'Selected tests' failed at 2023-06-14 20:16:52.729
         Executed 11 tests, with 4 failures (0 unexpected) in 0.0 (0.0) seconds

@FranzBusch
Copy link
Member Author

@swift-ci please test

@@ -54,3 +54,12 @@ extension String {
}

}

extension DefaultStringInterpolation {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not really happy with this. I think we can do better here. Are we planning to add the String(format) init in this package?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ultimately we will have access to all the interpolation-based formatters, so this seems like an appropriate approach.

@FranzBusch
Copy link
Member Author

With the latest changes I think we only have a problem on distantPast we are not getting 0000-01-01 but 0000-12-30. I guess we are missing some leap days here. @Lukasa is that expected from the implementation?

@Lukasa
Copy link

Lukasa commented Jun 15, 2023

That would have to be a lot of leap days to end up adding up to 12 months. What's the numerical value of distantPast?

@FranzBusch
Copy link
Member Author

That would have to be a lot of leap days to end up adding up to 12 months. What's the numerical value of distantPast?

It is -62135769600.0 from 1970

@FranzBusch
Copy link
Member Author

I got a bit further in investigating this the output of our Swift calculation is correct and aligns with gmtime; however, it does not align with what the current Date.description does on Darwin. I have aligned the tests in my recent commit but we need to discuss if we can change this since it produces semantically different results.

@itingliu
Copy link
Contributor

itingliu commented Jun 15, 2023

As far as I know gmtime doesn't seem to promise correctness before Gregorian calendar came into effect. That would also explain why we're only seeing test failures on distantPast, which is before Gregorian. If we can verify what the existing behavior is -- say, it's in fact formatting the date with Julian calendar -- I think we'd be more confident with this change even if it does cause a behavioral change.

@parkera parkera requested a review from itingliu June 20, 2023 20:39
@parkera
Copy link
Member

parkera commented Jun 20, 2023

It may be worth it to verify that distantPast (and future) exactly matches an existing output just by special casing it.

NOTICE.txt Outdated
* LICENSE (MIT):
* https://git.musl-libc.org/cgit/musl/tree/COPYRIGHT
* HOMEPAGE:
* https://musl.libc.org
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't seem standard to have a NOTICE.txt file in our other Swift repos. We have a license, which is reviewed for correctness.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do have a NOTICE.txt file in a bunch of repos for when we use other work or have to attribute it. Some examples:

@@ -0,0 +1,292 @@
//===----------------------------------------------------------------------===//
//
// This source file is part of the Swift Collections open source project
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem correct

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You are right. I copied the header from another file and it turns out that a bunch of files in here have the wrong copyright header.

@itingliu
Copy link
Contributor

As I chat with @FranzBusch I realized that this would be a huge behavioral change for Darwin users who don't use Gregorian calendar -- previously we use the calendar of their current locale, but now we'd be using Gregorian unconditionally. I personally don't think it's a good idea, but I'm curious to hear what others think.

Also we took a look into CFDateFormatter's implementation, and it did look dates before Oct 15, 1582 are formatted with Julian calendar. I think it's fine to change the description for these dates to be in Gregorian if we can just clarify this in comments.

@FranzBusch
Copy link
Member Author

As I chat with @FranzBusch I realized that this would be a huge behavioral change for Darwin users who don't use Gregorian calendar -- previously we use the calendar of their current locale, but now we'd be using Gregorian unconditionally. I personally don't think it's a good idea, but I'm curious to hear what others think.

Just to put some real outputs from the current behaviour of Darwin here. I ran this code with a bunch of different calendars.

print(Calendar.current.identifier)
print("now: \(Date(timeIntervalSince1970: 1687339291.480619).description)")
print("distantPast: \(Date.distantPast.description)")
print("distantFuture: \(Date.distantFuture.description)")

These are the outputs:

gregorian
now: 2023-06-21 09:21:31 +0000
distantPast: 0001-01-01 00:00:00 +0000
distantFuture: 4001-01-01 00:00:00 +0000

buddhist
now: 2023-06-21 09:21:31 +0000
distantPast: 0001-01-01 00:00:00 +0000
distantFuture: 4001-01-01 00:00:00 +0000

coptic
now: 1739-10-14 09:21:31 +0000
distantPast: -0283-05-06 00:00:00 +0000
distantFuture: 3717-04-08 00:00:00 +0000

ethiopic
now: 2015-10-14 09:21:31 +0000
distantPast: -0007-05-06 00:00:00 +0000
distantFuture: 3993-04-08 00:00:00 +0000

hebrew
now: 5783-10-02 09:21:31 +0000
distantPast: 3761-04-16 00:00:00 +0000
distantFuture: 7761-03-23 00:00:00 +0000

islamic
now: 1444-12-03 09:21:31 +0000
distantPast: -0640-05-18 00:00:00 +0000
distantFuture: 3483-02-23 00:00:00 +0000

iso8601
now: 2023-06-21 09:21:31 +0000
distantPast: 0001-01-01 00:00:00 +0000
distantFuture: 4001-01-01 00:00:00 +0000

japanese
now: 2023-06-21 09:21:31 +0000
distantPast: 0001-01-01 00:00:00 +0000
distantFuture: 4001-01-01 00:00:00 +0000

persian
now: 1402-03-31 09:21:31 +0000
distantPast: -0621-10-09 00:00:00 +0000
distantFuture: 3379-10-12 00:00:00 +0000

@parkera @itingliu What do you all think we should do here? Should we back out the changes for Darwin and just do this on Linux and Windows?

@parkera
Copy link
Member

parkera commented Jun 21, 2023

I think the main risk to changing the behavior would be if some app:

  1. Puts date.description directly in a UI
  2. The user of that app has a non-gregorian Calendar set
  3. This change results in an apparent regression in behavior, although of course the app should have been using a formatter or format style in the first place

If someone was relying on programmatically parsing the value, then they would have likely failed anyway when the user had a non-gregorian calendar set.

It does also regress the behavior for a developer debugging their app using a non-gregorian calendar, in that the dates they see in their app will not match the ones they get when poing in lldb.

@itingliu
Copy link
Contributor

t does also regress the behavior for a developer debugging their app using a non-gregorian calendar, in that the dates they see in their app will not match the ones they get when poing in lldb.

I don't worry too much about breaking app UI. But this second point is what I was concerned about. It would be a sad regression in developer experience. Would it be possible to ask lldb to format it differently themselves since they're already doing so for other types?

@parkera
Copy link
Member

parkera commented Jun 21, 2023

That's possible, and this is actually one of the best types to do that for since it is a very simple memory layout. I'm not sure if lldb has the logic to format dates for other calendars, though, and I doubt they would also be able to read the AppleLocale default to know what the right setting is...

@FranzBusch
Copy link
Member Author

I leave this up to you two here, but if developer experience is really the only thing we are concerned about here than I am sure they can understand if they have to use a DateFormatter here. UTC formatted time is pretty common for developers around the world.

I am happy to change the PR to add back the conditional for Darwin here if that makes it easier. It will just mean we have differing behaviour between the platforms. @parkera @itingliu your call :)

Copy link
Contributor

@itingliu itingliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know I'm back and forth on this -- but now I'm convinced that the performance gain from this change might outweigh the description being unlocalised. Let's give it a shot!

Sources/FoundationEssentials/Date.swift Outdated Show resolved Hide resolved
Sources/FoundationEssentials/Date.swift Outdated Show resolved Hide resolved
Sources/FoundationEssentials/TimeCalculations.swift Outdated Show resolved Hide resolved
Sources/FoundationEssentials/TimeCalculations.swift Outdated Show resolved Hide resolved
Sources/FoundationEssentials/TimeCalculations.swift Outdated Show resolved Hide resolved
assert((0...31).contains(date.day))
assert((0..<24).contains(date.hours))
assert((0..<60).contains(date.minutes))
assert((0..<61).contains(date.seconds))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are some those ClosedRange while the others Ranges?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code is coped over from swift-certificates but I assume this was chosen for readability otherwise we would have to use values such as 23 or 30 where it might be confusing. After reading these asserts I think the author chose the right trade off here.

Sources/_FoundationInternals/String/String+Internals.swift Outdated Show resolved Hide resolved
Sources/_FoundationInternals/String/String+Internals.swift Outdated Show resolved Hide resolved
@FranzBusch FranzBusch requested a review from itingliu June 29, 2023 13:31
@itingliu
Copy link
Contributor

@swift-ci please test

Copy link
Contributor

@itingliu itingliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This Linux build failure looks legitimate:

[587/689] Compiling FoundationEssentials TimeCalculations.swift
/build/swift-foundation/Sources/FoundationEssentials/TimeCalculations.swift:151:21: error: type 'Int64' has no member 'daysInMonth'
        while Int64.daysInMonth(months) <= remdays {
              ~~~~~ ^~~~~~~~~~~

@FranzBusch
Copy link
Member Author

@compnerd Could you test the latest commit of my branch on Windows?

@FranzBusch
Copy link
Member Author

@swift-ci please test

@compnerd
Copy link
Collaborator

compnerd commented Jul 6, 2023

@FranzBusch sorry for the delay, I was travelling. This doesn't seem to address all the problems:

I had to apply the following patch:

diff --git a/Sources/FoundationEssentials/TimeCalculations.swift b/Sources/FoundationEssentials/TimeCalculations.swift
index 481682a..62a4aed 100644
--- a/Sources/FoundationEssentials/TimeCalculations.swift
+++ b/Sources/FoundationEssentials/TimeCalculations.swift
@@ -148,8 +148,8 @@ extension Int64 {
         var years = remyears + (4 &* qCycles) + (100 &* cCycles) + (400 &* qcCycles)
 
         var months = 0
-        while daysInMonth(months) <= remdays {
-            remdays -= daysInMonth(months)
+        while Int64.daysInMonth(months) <= remdays {
+            remdays -= Int64.daysInMonth(months)
 
             // Unchecked because daysInMonth will crash if given a value greater than 12, so this
             // cannot exceed 12.

Test Suite 'Selected tests' started at 2023-07-05 17:37:54.430
Test Suite 'DateTests' started at 2023-07-05 17:37:54.435
Test Case 'DateTests.testDateComparison' started at 2023-07-05 17:37:54.435
Test Case 'DateTests.testDateComparison' passed (0.002 seconds)
Test Case 'DateTests.testDateMutation' started at 2023-07-05 17:37:54.437
Test Case 'DateTests.testDateMutation' passed (0.0 seconds)
Test Case 'DateTests.testDescription1970' started at 2023-07-05 17:37:54.437
S:\SourceCache\swift-foundation\Tests\FoundationEssentialsTests\DateTests.swift:90: error: DateTests.testDescription1970 : XCTAssertEqual failed: ("1970-01-01 00:00:00 +0000") is not equal to ("1939-1-1 0:0:0 +0000") -
Test Case 'DateTests.testDescription1970' failed (0.001 seconds)
Test Case 'DateTests.testDescriptionBeyondDistantFuture' started at 2023-07-05 17:37:54.438
Test Case 'DateTests.testDescriptionBeyondDistantFuture' passed (0.0 seconds)
Test Case 'DateTests.testDescriptionBeyondDistantPast' started at 2023-07-05 17:37:54.438
Test Case 'DateTests.testDescriptionBeyondDistantPast' passed (0.0 seconds)
Test Case 'DateTests.testDescriptionDistantFuture' started at 2023-07-05 17:37:54.439
S:\SourceCache\swift-foundation\Tests\FoundationEssentialsTests\DateTests.swift:102: error: DateTests.testDescriptionDistantFuture : XCTAssertEqual failed: ("4001-01-01 00:00:00 +0000") is not equal to ("3970-1-1 0:0:0 +0000") -
Test Case 'DateTests.testDescriptionDistantFuture' failed (0.0 seconds)
Test Case 'DateTests.testDescriptionDistantPast' started at 2023-07-05 17:37:54.439
S:\SourceCache\swift-foundation\Tests\FoundationEssentialsTests\DateTests.swift:97: error: DateTests.testDescriptionDistantPast : XCTAssertEqual failed: ("0000-12-30 00:00:00 +0000") is not equal to ("-31-12-30 0:0:0 +0000") -
Test Case 'DateTests.testDescriptionDistantPast' failed (0.0 seconds)
Test Case 'DateTests.testDescriptionReferenceDate' started at 2023-07-05 17:37:54.439
S:\SourceCache\swift-foundation\Tests\FoundationEssentialsTests\DateTests.swift:84: error: DateTests.testDescriptionReferenceDate : XCTAssertEqual failed: ("2001-01-01 00:00:00 +0000") is not equal to ("1970-1-1 0:0:0 +0000") -
Test Case 'DateTests.testDescriptionReferenceDate' failed (0.001 seconds)
Test Case 'DateTests.testDistantFuture' started at 2023-07-05 17:37:54.440
Test Case 'DateTests.testDistantFuture' passed (0.0 seconds)
Test Case 'DateTests.testDistantPast' started at 2023-07-05 17:37:54.440
Test Case 'DateTests.testDistantPast' passed (0.001 seconds)
Test Case 'DateTests.test_now' started at 2023-07-05 17:37:54.441
Test Case 'DateTests.test_now' passed (0.0 seconds)
Test Suite 'DateTests' failed at 2023-07-05 17:37:54.441
         Executed 11 tests, with 4 failures (0 unexpected) in 0.005 (0.005) seconds
Test Suite 'Selected tests' failed at 2023-07-05 17:37:54.441
         Executed 11 tests, with 4 failures (0 unexpected) in 0.005 (0.005) seconds

@FranzBusch
Copy link
Member Author

@compnerd Interesting. Not really sure where this difference is coming from since the Swift code is fully platform independent now. I sadly, don't have a Windows machine where I can reproduce this right now.

@parkera
Copy link
Member

parkera commented Sep 5, 2023

See also #252

@FranzBusch
Copy link
Member Author

@swift-ci please test

@parkera
Copy link
Member

parkera commented Oct 5, 2023

We're getting closer to being able to just use Foundation's own date/time calculations for this.

@FranzBusch
Copy link
Member Author

We're getting closer to being able to just use Foundation's own date/time calculations for this.

Great to hear. Maybe we can re-use this code then for the ICU-free time calculations.

@parkera
Copy link
Member

parkera commented Feb 5, 2024

@FranzBusch What do you think about refactoring this on top of the newly landed Gregorian calendar and ISO8601 formatting changes?

# Motivation

The current implementation of `Date.description` is depending on the what platform we are running on. On Darwin it used the CFFoundation backed implementation and on Linux/Windows it used libc APIs to do the date calculations. This caused build issues on Windows and the libc APIs are not 32bit clean.

# Changes

This PR changes the implementation to use the `FoundationEssentials` gregorian calendar and ISO 8601 formatter.

# Result

Swift only implementation that works cross platform for `Date.description` which also should perform better than any libc API calls.
@FranzBusch
Copy link
Member Author

@parkera I updated the code to use the new ISO801 formatter in FoundationEssentials and it works; however, the expected format is slightly different

("4001-01-01 00:00:00 +0000") is not equal to ("4001-01-01 00:00:00Z")

I don't see an option to configure the time zone formatting to align with the current formatting of Date.description. We have three options here:

  • Break the format of Date.description slightly
  • Copy the formatting code from the ISO8601 formatter and simplify/adapt it for Date.description
  • Extend the ISO801 format styles (this probably needs an evolution proposal)

What would you like to do?

@itingliu
Copy link
Contributor

itingliu commented Feb 12, 2024

According to spec

If the time is in UTC, add a Z directly after the time without a space...(omitted) A time zone where the civil time coincides with UTC is always designated as positive, though the offset is zero (see related specifications below). So the zone designation for London (on standard time) would be "+00:00", "+0000", or "+00".

But currently I don't think we distinguish between UTC and other GMT+0 timezones. These timezones would compare equal to each other

 17> TimeZone(identifier: "GMT+0") == TimeZone(identifier: "UTC")
$R6: Bool = true

This means that we cannot decide whether to use "Z" or use "+0000" by the time zone instance. And since we cannot tell those apart, I'm inclined to just expose the option directly publicly. But in the short run I think it's ok to expose an internal setting for Date.description

@parkera
Copy link
Member

parkera commented Feb 12, 2024

I agree with @itingliu -- we can add an internal-only option to the format style to prefer +0000 to Z. If it's interesting for other use cases beyond our one esoteric one we could pitch it as public API too.

@parkera
Copy link
Member

parkera commented Feb 12, 2024

This line would just need an additional if:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants