station_server: provide history handlers #936

karlp · 2020-06-12T17:29:50Z

While the history handlers do need to match what has been saved on disk,
the project provides MfgEvent protobuffer writers out of the box, and
no-others, so at least let the out of box experience function as
expected, even if you would want to change this in your own
implementations.

To enable the (built in) writers, something like this is required in
your station server.


if __name__ == '__main__':
	openhtf.util.conf.load(station_server_port='4444')

	interface = mfg_inspector.MfgInspector()
	interface.set_converter(mfg_event_from_test_record)

	with station_server.StationServer(history_path="somepath") as server:
		while 1:
			test = .... #your tests here
			test.add_output_callbacks(server.publish_final_state)
			# explicitly match hardcoded pattern in HistoryListHandler
			test.add_output_callbacks(interface.save_to_disk("somepath/mfg_event_{dut_id}_{start_time_millis}.pb"))
			test.execute(test_start=user_input.prompt_for_test_start())

Signed-off-by: Karl Palsson [email protected]

This change is

googlebot · 2020-06-12T17:29:54Z

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

What to do if you already signed the CLA

Individual signers

It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

arsharma1 · 2020-06-26T14:15:07Z

Hi Karl,
Please accept the CLA, which is needed before I can look at and comment on this PR.

Thanks!

karlp · 2020-06-26T14:42:18Z

Now that it appears that maintainers might still exist I can look at that :) The GCLA is a bit more paperwork than the eclipse one I wasn't goign to waste time on it given the general activity level of the project earlier :)

karlp · 2020-08-18T09:11:31Z

@googlebot I signed it!

google-cla · 2020-08-18T09:11:35Z

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

karlp · 2020-08-19T11:40:50Z

@arsharma1 CLA is now signed

arsharma1 · 2020-06-12T18:34:26Z

openhtf/output/servers/station_server.py

+    with open(fn, mode="rb") as f:
+      me = mfg_event_pb2.MfgEvent()
+      me.ParseFromString(f.read())
+      from openhtf.core.test_record import TestRecord


Move imports to the top of the file.

Additionally, please do not import classes (or functions) out of modules; just import the module directly.

from openhtf.core import test_record

have just dropped it, it was only needed for the type annotation

arsharma1 · 2020-06-12T18:37:30Z

openhtf/output/servers/station_server.py

+      me = mfg_event_pb2.MfgEvent()
+      me.ParseFromString(f.read())
+      from openhtf.core.test_record import TestRecord
+      tr: TestRecord = mfg_event_converter.test_record_from_mfg_event(me)


PY2 is still technically supported for now, so please do not add type annotations.

I hope to drop PY2 support in the next few months as I really want annotations everywhere.

arsharma1 · 2020-06-12T18:39:23Z

openhtf/output/servers/station_server.py

@@ -463,8 +465,25 @@ class HistoryItemHandler(BaseHistoryHandler):
  def get(self, file_name):
    # TODO(Kenadia): Implement the history item handler. The implementation


please remove this TODO as you're fixing it!

arsharma1 · 2020-06-12T18:39:33Z

openhtf/output/servers/station_server.py

@@ -479,8 +498,23 @@ class HistoryAttachmentsHandler(BaseHistoryHandler):
  def get(self, file_name, attachment_name):
    # TODO(Kenadia): Implement the history item handler. The implementation


remove TODO

arsharma1 · 2020-06-12T18:40:20Z

openhtf/output/servers/station_server.py

+    with open(fn, mode="rb") as f:
+      me = mfg_event_pb2.MfgEvent()
+      me.ParseFromString(f.read())
+      # TODO - could use sha1 here to check?


TODO:

This allows our TODO search tools to identify it.

arsharma1 · 2020-06-12T18:42:13Z

openhtf/output/servers/station_server.py

+
+    fn = os.path.join(self.history_path, file_name)
+    if not os.path.isfile(fn):
+      self.write("Not found")


Please use single quotes for strings.

May have missed this comment.

arsharma1 · 2020-06-12T18:42:55Z

openhtf/output/servers/station_server.py

+        self.write(desired_real[0].value_binary)
+        self.set_status(200)
+      else:
+        self.write("some, but no match?!")


self.write('Attachment not found in test.')

google-cla · 2020-08-19T16:47:12Z

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

karlp · 2020-08-19T16:47:14Z

Updated thanks for the review

arsharma1 · 2020-08-27T17:40:49Z

openhtf/output/servers/station_server.py

+      self.set_status(404)
+      return
+
+    with open(fn, mode="rb") as f:


Please use single quotes for strings.

arsharma1 · 2020-08-27T17:41:09Z

openhtf/output/servers/station_server.py

+      self.set_status(404)
+      return
+
+    with open(fn, mode="rb") as f:


Single quotes.

arsharma1 · 2020-08-27T17:41:41Z

openhtf/output/servers/station_server.py

+
+    fn = os.path.join(self.history_path, file_name)
+    if not os.path.isfile(fn):
+      self.write("Not found")


May have missed this comment.

karlp · 2020-08-27T20:47:01Z

Yes, sorry, fixed on memory and missed all your comments :( will do better.

While the history handlers do need to match what has been saved on disk, the project provides MfgEvent protobuffer writers out of the box, and no-others, so at least let the out of box experience function as expected, even if you would want to change this in your own implementations. To enable the (built in) writers, something like this is required in your station server. ``` if __name__ == '__main__': openhtf.util.conf.load(station_server_port='4444') interface = mfg_inspector.MfgInspector() interface.set_converter(mfg_event_from_test_record) with station_server.StationServer(history_path="somepath") as server: while 1: test = .... #your tests here test.add_output_callbacks(server.publish_final_state) # explicitly match hardcoded pattern in HistoryListHandler test.add_output_callbacks(interface.save_to_disk("somepath/mfg_event_{dut_id}_{start_time_millis}.pb")) test.execute(test_start=user_input.prompt_for_test_start()) ``` Signed-off-by: Karl Palsson <[email protected]>

google-cla · 2020-08-28T13:37:05Z

CLAs look good, thanks!

ℹ️ Googlers: Go here for more info.

karlp · 2020-08-28T13:37:13Z

updated thanks!

karlp · 2021-08-27T11:26:58Z

Anything still blocking this?

karlp · 2022-07-05T14:28:54Z

Ok, it's approved, is there anything else to do here?

willeccles · 2023-05-23T18:16:49Z

I have applied this to my fork and tested. It works for the most part, but the MIME types of attachments are not respected.

willeccles · 2023-05-23T18:38:38Z

Seems to be fixed with the following patch:

diff --git a/openhtf/output/servers/station_server.py b/openhtf/output/servers/station_server.py
index 260fbc7..50c86ee 100644
--- a/openhtf/output/servers/station_server.py
+++ b/openhtf/output/servers/station_server.py
@@ -537,6 +537,7 @@ class HistoryAttachmentsHandler(BaseHistoryHandler):
       # TODO: could use sha1 here to check?
       desired_real = [a for a in me.attachment if a.name == attachment_name]
       if len(desired_real) > 0:
+        self.set_header('Content-Type', desired_real[0].mime_type)
         self.write(desired_real[0].value_binary)
         self.set_status(200)
       else:

karlp mentioned this pull request Jun 26, 2020

StationServer doesn't record history #935

Open

arsharma1 reviewed Aug 19, 2020

View reviewed changes

karlp force-pushed the history-handlers branch from d12634e to b79fc66 Compare August 19, 2020 16:46

arsharma1 reviewed Aug 27, 2020

View reviewed changes

karlp force-pushed the history-handlers branch from b79fc66 to 5ba152a Compare August 28, 2020 13:37

karlp mentioned this pull request Sep 1, 2020

feature request: attachments in webgui as links/downloadable files #823

Open

arsharma1 requested a review from cricdecyan August 27, 2021 19:50

cricdecyan approved these changes Aug 27, 2021

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

station_server: provide history handlers #936

station_server: provide history handlers #936

karlp commented Jun 12, 2020 •

edited by kdsudac

Loading

googlebot commented Jun 12, 2020

arsharma1 commented Jun 26, 2020

karlp commented Jun 26, 2020

karlp commented Aug 18, 2020

google-cla bot commented Aug 18, 2020

karlp commented Aug 19, 2020

arsharma1 Jun 12, 2020

karlp Aug 19, 2020

arsharma1 Jun 12, 2020

arsharma1 Jun 12, 2020

arsharma1 Jun 12, 2020

arsharma1 Jun 12, 2020

arsharma1 Jun 12, 2020

arsharma1 Aug 27, 2020

arsharma1 Jun 12, 2020

google-cla bot commented Aug 19, 2020

karlp commented Aug 19, 2020

arsharma1 Aug 27, 2020

arsharma1 Aug 27, 2020

arsharma1 Aug 27, 2020

karlp commented Aug 27, 2020

google-cla bot commented Aug 28, 2020

karlp commented Aug 28, 2020

karlp commented Aug 27, 2021

karlp commented Jul 5, 2022

willeccles commented May 23, 2023

willeccles commented May 23, 2023

		@@ -463,8 +465,25 @@ class HistoryItemHandler(BaseHistoryHandler):
		def get(self, file_name):
		# TODO(Kenadia): Implement the history item handler. The implementation

		@@ -479,8 +498,23 @@ class HistoryAttachmentsHandler(BaseHistoryHandler):
		def get(self, file_name, attachment_name):
		# TODO(Kenadia): Implement the history item handler. The implementation

station_server: provide history handlers #936

Are you sure you want to change the base?

station_server: provide history handlers #936

Conversation

karlp commented Jun 12, 2020 • edited by kdsudac Loading

googlebot commented Jun 12, 2020

What to do if you already signed the CLA

Individual signers

Corporate signers

arsharma1 commented Jun 26, 2020

karlp commented Jun 26, 2020

karlp commented Aug 18, 2020

google-cla bot commented Aug 18, 2020

karlp commented Aug 19, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TODO:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

google-cla bot commented Aug 19, 2020

karlp commented Aug 19, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

karlp commented Aug 27, 2020

google-cla bot commented Aug 28, 2020

karlp commented Aug 28, 2020

karlp commented Aug 27, 2021

karlp commented Jul 5, 2022

willeccles commented May 23, 2023

willeccles commented May 23, 2023

karlp commented Jun 12, 2020 •

edited by kdsudac

Loading