Sublime Forum

Retrieving inserted and deleted text

#21

I took at shot at this using the ideas mentioned from @fico, @kingkeith, and I. I am not a python programmer so I may be doing some things incorrectly:

import sublime, sublime_plugin

class StorytellerListener (sublime_plugin.EventListener):

	def __init__(self):

		#do we need to call the base class constructor???

		#a shadow copy of the text in the buffer (this will be changed to event 
		#objects with timestamp, dev group, etc. later)

		#this object will have a 2D array for each buffer (keyed by the buffer id)
		#there will be an arry for each line and events will be added to a line
		#newlines will be added to the end of the line they are on
		#file 1:
		#123
		#456
		#789
		#[["1", "2", "3", "\n"]["4", "5", "6", "\n"]["7", "8", "9"]]
		self.allInsertEventsByFile = {}

		#stores the size of each buffer (keyed by the buffer id)
		self.allBufferSizes = {}

	def on_new(self, view):

		#all new files get an empty 2D array to hold insert events
		self.allInsertEventsByFile[view.buffer_id()] = [[]]
		
		#all new files start with a buffer size of 0
		self.allBufferSizes[view.buffer_id()] = 0

	#view is the changed file
	def on_modified (self, view):		

		#get all of the cursors (this only works with one)
		updatedRegions = view.sel()

		#go through each of the regions (usually only one- multiple regions if 
		#there are multiple cursors)
		for region in updatedRegions:

			#get the current size of the text inside the active buffer
			newBufferSize = view.size()

			#get the id of the buffer for a multiple file project
			bufferId = view.buffer_id();

			#holds the number of characters that were either inserted or deleted
			numCharsChanged = abs(newBufferSize - self.allBufferSizes[bufferId])

			#if the current buffer size is exactly the same as the last edit 
			if(newBufferSize == self.allBufferSizes[bufferId]):

				#I'm not sure when this happens!!!
				print("Size the same-- When does this happen????")

			#new buffer size is smaller than the old buffer size, this must be a delete
			elif(newBufferSize < self.allBufferSizes[bufferId]):

				#remove some data from the shadow container
				self.removeEvent(numCharsChanged, region.a, self.allInsertEventsByFile[bufferId], view)

			#buffer is larger, must be an insert
			else:
				self.insertEvent(numCharsChanged, region.a, self.allInsertEventsByFile[bufferId], view)

			#store the new buffer size for next time
			self.allBufferSizes[bufferId] = newBufferSize


	def printEvents(self, view):
		
		print("buffer id: %i" % view.buffer_id())

		#get the events for this buffer
		allEvents = self.allInsertEventsByFile[view.buffer_id()]

		#go through all of the rows
		for row in allEvents:

			#print each row
			print(row)


	def insertEvent(self, numCharsChanged, cursorPoint, bufferOfInsertEvents, view):

		#get the region with the new text
		insertedRegion = sublime.Region(cursorPoint - numCharsChanged, cursorPoint)

		#get the text from that region of the view
		insertedText = view.substr(insertedRegion)

		#go back to where the insert started and get the row and column
		beginInsertRow, beginInsertColumn = view.rowcol(cursorPoint - numCharsChanged)

		#used to specify where to insert
		insertRow = beginInsertRow
		insertColumn = beginInsertColumn

		#for each of the new characters, insert into the shadow buffer
		for newChar in insertedText:

			#if the new character is a newline
			if newChar == "\n":

				#get the rest of the current line so we can add it to the next line
				restOnLine = bufferOfInsertEvents[insertRow][insertColumn:]

				#insert the newline event where it occured in the current line
				bufferOfInsertEvents[insertRow].insert(insertColumn, newChar)

				#insert a new line and add the rest of the previous line  
				bufferOfInsertEvents.insert(insertRow + 1, restOnLine)

				#remove all of the characters we already added
				for x in restOnLine:

					#remove from the end of the line
					bufferOfInsertEvents[insertRow].pop()

				#start back at the beginning of the next line
				insertRow = insertRow + 1
				insertColumn = 0 
			
			else: #the new character is NOT a newline

				#if the insert is on a new line (happens after deleting all the 
				#chars on a line and then adding some back)
				if insertRow == len(bufferOfInsertEvents):

					#add the new line
					bufferOfInsertEvents.append([])

				#add the character event to the current line
				bufferOfInsertEvents[insertRow].insert(insertColumn, newChar)

				#move the column index forward
				insertColumn = insertColumn + 1						

		#print the new state of the events
		self.printEvents(view)


	def removeEvent(self, numCharsChanged, cursorPoint, bufferOfInsertEvents, view):

		#get the row and column of the cursor after the delete (at the beginning of the delete)
		deleteRow, deleteColumn = view.rowcol(cursorPoint)

		count = 0

		#start removing events, once for each of the chars deleted
		while count < numCharsChanged:

			#if we are removing a newline
			if bufferOfInsertEvents[deleteRow][deleteColumn] == "\n":
				
				#remove the newline from the end of the line
				del bufferOfInsertEvents[deleteRow][deleteColumn]

				#if there are more rows underneath
				if deleteRow < len(bufferOfInsertEvents) - 1:

					#grab the next line
					nextLine = bufferOfInsertEvents[deleteRow + 1]

					#add the characters from the next line to the current line
					for char in nextLine:
						bufferOfInsertEvents[deleteRow].append(char)

					#remove the next line since we copied it over
					bufferOfInsertEvents.pop(deleteRow + 1)

			else: #removing a non-newline

				#all removes happen at the cursor point since and events slide back
				del bufferOfInsertEvents[deleteRow][deleteColumn]

				#if the line is completely empty now AND it is not last line
				if len(bufferOfInsertEvents[deleteRow]) == 0 and len(bufferOfInsertEvents) > 1:

					#remove the empty line
					bufferOfInsertEvents.pop(deleteRow)
				
				#if we just removed the last character on a line
				elif deleteColumn == len(bufferOfInsertEvents[deleteRow]):

					#move on to the next row
					deleteColumn = 0
					deleteRow = deleteRow + 1

			count = count + 1

		#print the new state of the events
		self.printEvents(view) 

Currently, it only tracks new files. To run it, open up a new file using sublime and make sure the console is open.

It seems to work with single cursor inserts and deletes (typing and cutting and pasting). I am using the buffer size to determine if there was an insert or a delete. I am not sure when the file would change and the buffer size would be exactly the same size as before but it does appear to happen occasionally. Does anyone know when/why that happens?

I am also not sure about find/replace scenarios, multiple cursors, etc. If anyone with a lot of sublime experience can guide me that would be much appreciated.

2 Likes

#22

Lookin good :grin:

I messed around with it for a few minutes, didnā€™t figure anything major out but did manage to take out a traceback caused by allBufferSizes being uninitialized.

Code @ Gist

Ā 



Ā 
I wanted to figure out the multi-caret thing but need to get back to working on another project.

I think the math involving numCharsChanged & cursorPoint needs to be re-worked. Ā  Currently it is lumping all of the modifications to a single cursor point. Ā  In addition to numCharsChanged, offsets of each cursor need to be taken into consideration.
Ā 



Ā 
You should host this on GitHub. Ā  Iā€™d definitely follow it & contribute a bit when I have some extra time.

It would be fun to get that set_timeout_async script working as a ST player for your data files.

1 Like

#23

You donā€™t need to use lambda, but to call a function with arguments, you need to do something special to wrap the function call in another function, so that it will remember the parameters without immediately executing the function. i.e.

import sublime, sublime_plugin

global string, stringIndex

class TestCommand ( sublime_plugin.TextCommand ):
    def run ( self, edit ):
        global string, stringIndex
        string = "Testing delayed output."
        stringIndex = 0
        
        def run_timeout_insert():
            self.view.run_command ( "timeout_insert" )
        
        for index in range ( 0, len ( string ) ):
            sublime.set_timeout_async( run_timeout_insert, 100 * index )

class TimeoutInsertCommand ( sublime_plugin.TextCommand ):
    def run ( self, edit ):
        global string, stringIndex
        self.view.insert ( edit, self.view.sel()[0].a, string[ stringIndex ] )
        stringIndex += 1

or, more concisely, as the insert command is built in and wrapping a function (to ā€œcreate a closureā€ I believe is the term) is a common operation, Python has a built in method for it:

import sublime, sublime_plugin, functools

class TestCommand ( sublime_plugin.TextCommand ):
    def run ( self, edit ):
        string = 'Testing delayed output.'
        
        for index in range ( 0, len ( string ) ):
            sublime.set_timeout_async( functools.partial(self.view.run_command, 'insert', args = { 'characters' : string[index] } ), 100 * index )

see this post on SO for more details

3 Likes

#24

My proof of concept, will log changes to the console: (EDIT: just edited it to work with more than one cursor)

import sublime, sublime_plugin

def get_cursors(view):
    return [cursor for cursor in view.sel()] # can't use `view.sel()[:]` because it gives an error `TypeError: an integer is required`

class RecordSessionListener(sublime_plugin.EventListener):
    prev_cursors = {}
    
    def on_new_async(self, view):
        self.record_cursor_pos(view)
    
    def on_activated_async(self, view):
        self.record_cursor_pos(view)
    
    def record_cursor_pos(self, view):
        cursors = []
        for cursor in get_cursors(view):
            if cursor.empty() and cursor.begin() > 0: # if the cursor is empty and isn't at the start of the document
                cursors.append((cursor, view.substr(cursor.begin() - 1))) # record the previous character for backspace purposes
            else:
                cursors.append((cursor, view.substr(cursor))) # record the text inside the cursor
        
        self.prev_cursors[view.id()] = cursors
    
    def on_selection_modified_async(self, view):
        self.record_cursor_pos(view)
    
    def on_insert(self, view, cursor_begin, cursor_end, text):
        self.log('insert', 'from', view.rowcol(cursor_begin), 'to', view.rowcol(cursor_end), '"' + text + '"')
    
    def on_delete(self, view, cursor_begin, cursor_end, text):
        self.log('delete', 'from', view.rowcol(cursor_begin), 'to', cursor_end, '"' + text + '"')
    
    def log(self, *values):
        print(type(self).__name__, *values)
    
    def on_modified_async(self, view):
        offset = 0
        for index, cursor in enumerate(view.sel()):
            prev_cursor, prev_text = self.prev_cursors[view.id()][index]
            prev_cursor = sublime.Region(prev_cursor.begin() + offset, prev_cursor.end() + offset)
            if not prev_cursor.empty() or cursor.begin() < prev_cursor.begin():
                self.on_delete(view, prev_cursor.begin(), prev_cursor.end(), prev_text)
            if cursor.begin() > prev_cursor.begin():
                region = prev_cursor.cover(cursor)
                self.on_insert(view, region.begin(), region.end(), view.substr(region))
            offset += cursor.begin() - prev_cursor.begin()

works with cuts, pastes etc. although it doesnā€™t seem to handle undo very well at the moment, it calls it the opposite of what it isā€¦ but should give you a clear starting point, you can just create a class that extends RecordSessionListener and make use of the on_insert and on_delete methods :slight_smile:

3 Likes

Clear non-saved state when changes are reverted manually
Count the no of times a user used the suggestions given by the autocomplete
#25

Ā 
Awesome, thanks for clarifying!

The functools.partial method is too intense for me :dizzy_face: lol. Ā  I like my whitespace & verbose declarations, keeps me from having to think too much while scanning through code.

1 Like

#26

I made a slight change to the on_modified this morning that should handle multiple cursors:

	def on_modified (self, view):		

		#get all of the cursors (this only works with one)
		updatedRegions = view.sel()
		
		#get the current size of the text inside the active buffer
		newBufferSize = view.size()

		#get the id of the buffer for a multiple file project
		bufferId = view.buffer_id();

		#if we are tracking this view
		if bufferId in self.allBufferSizes and bufferId in self.allInsertEventsByFile:
			
			#holds the number of characters that were either inserted or deleted
			numCharsChanged = abs(newBufferSize - self.allBufferSizes[bufferId])

			#MM- change here!!
			#if all of the changes are exactly the same length we can handle
			#them all
			numCharsChanged = numCharsChanged / len(updatedRegions)

			#go through each of the regions (usually only one- multiple regions if 
			#there are multiple cursors)
			for region in updatedRegions:
				
				#if the current buffer size is exactly the same as the last edit 
				if(newBufferSize == self.allBufferSizes[bufferId]):

					#I'm not sure when this happens!!!
					print("Size the same-- When does this happen????")

				#new buffer size is smaller than the old buffer size, this must be a delete
				elif(newBufferSize < self.allBufferSizes[bufferId]):

					#remove some data from the shadow container
					self.removeEvent(numCharsChanged, region.a, self.allInsertEventsByFile[bufferId], view)

				#buffer is larger, must be an insert
				else:
					self.insertEvent(numCharsChanged, region.a, self.allInsertEventsByFile[bufferId], view)

			#store the new buffer size for next time
			self.allBufferSizes[bufferId] = newBufferSize

This works only if all of the changes being added are exactly the same length (like in a multi-cursor situation). I take the difference in buffer size and divide by the number of cursors.Three instances of inserting the word ā€˜catā€™ would have a difference of 9 characters. Divide by the three cursors and each cursor contributes 3 characters. Send the position of each cursor to the insert/delete function and it seems to work.

This will not work if unequal sized chunks of text are ever entered at multiple points by the tool. Iā€™m not sure if that ever happens in sublime. Some other editors are more transparent about which changes are occurring.

I also checked and found that find and replace text of exactly the same length does not workā€¦ still thinking about how I might solve that problem.

Yes, if I end up going with sublime I will put it on github soon. I am still checking out a few other editors.

1 Like

#27

Very nice!

1 Like

#28

Nice! Ā  Glad I asked what you were using that code for, this is a really cool project :smiley:

1 Like

#29

@fico @kingkeith I decided to go with visual studio code for my more complete version. Sublime is awesome but I wanted to reuse what I had in js. Here is the project page: https://github.com/markm208/storyteller#storyteller

I am curious for your input!

3 Likes

#30

wow, that is really cool! Iā€™m exclusively an Sublime Text user, so I havenā€™t tried the plugin itself, but the playback mode is awesome! great work! Iā€™m jealous that you didnā€™t write the storyteller extension for Sublime Text :wink:

1 Like

#31

Works great, nice job man! :grin:

Ā 

Suggestion:

  • Add an icon-based UI group for controls

Ā 

The first time I tried it out, I couldnā€™t tell that there was a speed setting. Then when I skimmed through the documentation, I noticed the keyboard commands.

A control panel can add visual indication of the current speed, and IMO looks better & is more intuitive than text buttons (paired with hover-tooltips for context).

Ā 

Here are a few mockups showing potential states of said UI:
#@Imgur

Ā 


Ā 

Edit:

I didnā€™t notice the StepBackward & StepForward buttons when I made the mockups, so hereā€™s an adjusted one:

1 Like

#32

Thanks for posting this. How did you discover those sublime_plugin.EventListener.on_insert, on_delete methods? This is not documented, is it?

1 Like

#33

the on_insert and on_delete methods are not a built in part of sublime_plugin.EventListener - I had to do some coding wizardry to determine what text was inserted and deleted and where - if you look closely, the code I posted uses on_selected_modified_async and on_modified_async together to determine this information - the idea being someone could extend my RecordSessionListener class which has already done all the hard work :wink:

1 Like

#34

Hey thanks for the answer. I should have seen that. Didnā€™t pay enough attention.

1 Like

#35

no worries - Iā€™ve done that (not pay enough attention) many times recently :wink:

1 Like

#36

Any idea about how to catch undo/redo (preferably multi-cursor too) modifications?

When a multi-cursor undo/redo modification occurs, the selection does not change to the selection inside the undo/redo action.

1 Like

#37

I donā€™t have time right now to have a look, but a horrible hack might be to listen for when an undo or redo command is about to be executed, cache the entire buffer, then after it is executed, compare the cache with the current bufferā€¦ terrible though, I knowā€¦

1 Like

#38

one relatively unknown little gem in ST is that you can create a keybinding that will operate on any character press:

import sublime
import sublime_plugin


class CharsInsertedCommand(sublime_plugin.TextCommand):
    def run(self, edit, character):
        print('character "{}" inserted at positions {}'.format(character, [sel for sel in self.view.sel()]))
        self.view.run_command('insert', { 'characters': character })

{ "keys": ["<character>"], "command": "chars_inserted" },

character ā€œtā€ inserted at positions [(0, 0)]
character ā€œeā€ inserted at positions [(1, 1)]
character ā€œsā€ inserted at positions [(2, 2)]
character ā€œtā€ inserted at positions [(3, 3)]

though doing such a thing could easily conflict with other plugins if they also use it, and may affect they way undo states are calculatedā€¦

1 Like

Block default keypess event programatically
#39

Here is what I want to achieve in my plugin.

  • Enter a special command (some text) in a ST tab
  • Press a key combination
  • Launch a subprocess and pass the entered text to the subprocessā€™ stdin.
  • Capture the output of the subprocess and paste it into the view next to the entered command (POSITION).

The main goal here is to NOT block the editor tab (view) while the subprocess is working. The user must be able to freely modify the viewā€™s buffer. The user also must be able to run many subprocesses.

To make this work I need to find a way to adjust the POSITION for each subprocess when the user modifies the view.

1 Like

#40

have you considered just using STā€™s built in bookmarks functionality? these seem to stay in place even when text before it is edited.

1 Like